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(54) MUSICAL-SOUND RESPONDING IMAGE GENERATION SYSTEM, METHOD AND DEVICE 
AND RECORDING MEDIUM THEREFOR 

(57)Abstract: 

PROBLEM TO BE SOLVED: To provide a joining type 
musical-sound responding image generation system 
which can set the movement optionally by generating a 
moving picture changing integrally as a musical sound is 
played. 

SOLUTION: A sequencer S supplies musical sound 
control information and a synchronizing signal 
corresponding to music to a sound source module A and 
an image source module I and a parameter setting 
submodule PS supplies operation parameters for 
controlling the movement of an image to the image 
source module I. The module I while referring to the 
operation parameters sequentially controls the 
operations of respective image parts in response to the 
musical-sound control information and synchronizing 
signal to generate an image which moves in synchronism 
with the progress of musical sound generation by a 
musical sound module A. Processes for prereading, 
interpolation, musical-sound control information analysis, 

and display switching are applied to this moving picture generation to generate a variety of 
moving images which synchronize with the musical sound playing more securely, move smoothly 
and naturally, and correspond to the played music. 
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(54) [K9i<D£m & ; %fo&m8ifiy&i'XTlx> fig, Xtf. *-©fc»©ffi®«Kt: 



(57) [gift] 

[Sill] ^SIIICDMfTi-ttttK^t-f SttiB&S: 

i Ft iicc mm l x x a - x x- e «stc»fp i, 

•5. 
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a. ac*. 

* - * «cse •? -ctt* #mhp ^ ti *BHk&£j$r * @u#*£ 10 

[ifl*Si2 ] ^»XifeCc#fti,Tj|»MaimRa 

MJHC, 20 

y - * sets o Tg-eucDSSir mmm n«B«*4«r s 

#>T J: 5 cc Lfc c t 4W«4T zmt&BWKSz&ti 

& 

hhr*w*ws ^i£sitf>©tit«45fctj!*oft? 

JttlCr*OTBH««r*lW 4 C 1 4*J8i<!:-f Sfl!#:ri2 
[itj#S4 ] ■W2H**SfefiH-*fca&©5/*?-A©*Bl 

4«f8i t r snsiagi 2 rcro©a&i&5H*£dt£&. 

M^5]tt«lllft&3IHHR*3N:U DtCXffiHfl 
t&&4£tc L/"C C ©*HHft*#© t <S"<»ai*»tt4W 

*4©-T -5 C <t *«Ni4f 4» 3 tCfBJfi©^ 
UiBjU^-r^t^ttCc^cEtr. ffl<tl©Sg&©!»*4lijl) 

*«. 
«. aw. 

nEawmwtwawiBBWfi^Ka^flr. wbimp** so 
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y-£Kfi£->rtb£ **M» 3 ti & Wk 4£j& * S 

fis^gcc «fc & *s£fijw»Mf «c -a l r n < mn«s£fis 

Sh4J:3«ci,fcCt 4f$® £ f &$tf{&SBHl£jffi8 

So 

WlMEffim? * *tt (Cttlfc 1/ "C IMP < 5 j* - dr 4 gyg 

u 

«8B3i»iw»tiMRacn^W(i*fc»-3#. mrsasbfp^ 
y - £ test o r &si5©Eb§ imm s n £HHR4£ifc $ tr 
r . «S^j«o«f K^to*r Iifl!i*tt36»-r J: 5 cc bfc 

7" P -fe X * JEff r €> fc«>© 7* p ^ ? A 4Ctt 0 

[*»i©mE&iKin 

[0001] 
[0002] 

97. (CG : computer qraphics ) {C«fc«9Bi^4S!{fc3 

>Ktry»T;U (BGV) -C*0. ^SbKlffiifflftSBI 
X9 3 « r *>' 9 , XSM Wff *tt i"4ffl t >r iii^4.# 

bif— vCC0/cfc©«. ^/c6T. -^-f -ir 

ffi*&ti-^-r*»r>fc. s etc. MiDijgs* 

[0 00 3]—*. ^57 -f ?XtcJ:6-rfciIflf-(» 

K«t»)«-T«S»*&©««»«©iaffl*WSL. C© 
ftSMCi&Cfc-feU' hfi^KJ:orWR©HiflU<j»-> 

nrus^©T. ^ £©^W«: f ^ ^ y ^ /cSb^E® 

[0004] £ fc. C©5Sg^Si^{t^g© J: ^{cjB®^' 
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[0005] 

Bfli&*£;#fiL-Cft3ftfcfc©-C\ C<D&W<D — 3 CDS 10 

ttti. midi *ft©«fc^ft^s£©Mf|{ci5i$L-aii3 

r 5 c 6#-c s 3 > e * - * y 7 7 

ib i ©-f*si(cffinfcKiiii^€r gam^t h it w r & 
* h©tt*t:x~ir:PS&tcR£-r£c£*>-c*s#in 

[000 6] Se.CC. *»^-dr *»»Cl,"CCG««* 

©sft*£D*sft#<fe*K fflra^RRtcit. =>> 
t»-£©CGiSiiififc*>-*>c p u^©:ftiwS«jK:«fc o r 

*^{istx-cL* c ttfi&ztcibic, mmomfticffl 

m utrc g 7- - ^ a >*ms-r 5 C <b ifi-C t ftt»C 
T&HKCtt, H»»»ft*«;*««5*-ir«:j6Grffl8ll 

fi5Lh©3g*tgK:&*. i?Tg®i{£©£Ji£©iSft 
ctOT«> ^^©^ffle&fttctcXTXA- 

© * i * ») *fl¥WT £ C £ to J: «3 HS** fvv £ g 
ttJHMrcttfr 3-tt£c£4>-C# £ tf 8Kt 

CiJC&S. 40 
[0007] 

[ifj!i*ft?ifc-rsfc«>©^f2] *»*»*BPB*»»r&fc 

£ CC?S o T «J t 3 ft £ ®Hf! ^iSKf -S Iffl®^^ 50 
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fgic J: ***^BS© jiR tc-« t r«j < BHft*<£ft s ft 
i. 

[ 0 0 0 8 ] * tc. C0&«®X*£fflft£J$#£tC 
LT «ifp< 9 y - 5 fcfltt&T S a » ?\ mCXtfHflP 

m«K:a-5(,»T«**4jssr*x^s»^. ewc. Bute 
* cc&o rssi5©«j * *sM9 § ft SHMfcfefibfcr ^ - 

<fc5CC3ft-S. 

[0009] c ©IfeBjccj: «X*j&Sflm£titir£<c*s 
^Kfet»-cm2«SIMWIMR© 5 £S3ROTMktt£jdt 

[ooio] c<o»w©a«©«F*K:j:*t. ffiaiam© 

U-ASrfiJfflL. *»*>S*-7U-AH©?ira0»«:-5' ! 
<!: (C J; o TSf jS(CH3K 3 ft 6. 

[0011] C©»TOS6KSij0^8l(CJ:5i, B^ifi 

©»*i«wap«*i&. 

[0012] c©|£Bj(c«fc2,lglff£;Slii**fefiS 
©fcaboEfflKfttcts. iH^-r^lriliflcc^iEixtrig^ 

r*»©»*#*(W3ft*IB«*£J«StfT. 

©jttf(c^to#rafii*sfc*-r J: 5tcL/c ^a^^«rii 
if r Sfc»©y a y 5 a >»sctt s ft * . 

[0013] 



* .* 
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£tCMI D I (Musical Instrumentdiqital Interfac 
e) Sl*^- mi$*-7i?x * h(CB. C©<£ 

(3D) Sj^i-rS©*s^SW-C*S. C©^icJ:S 

i. m i d u%m : ?-i*t,c-g$tiz>m : mimtt&*mw 

[00 15] CO^Bjrti. M I D I»»^-jr©J:9 

^ — !f ©sft5£«c J: o r fflti&V -> - ^ > x £ sfct-f & iii 

I D I jjSHf*— #T?fei^l/£f J: 5 (ctSci i>~C&2>. 20 
[ooie] co&Bjj-eii, § ^{ C m 1 d 1 
f - * te j: s^^inssof-e-tx o fclft^ L tf i 
i>5Ci<l:«ttt<c<. fflfflir. mat. #>x©J: 

(CMC^-If. 8&ft^-5y-£©ffi:g©ig3£SMK:<fc'). 
[0017] C OJti-CIJ. iStUT*- *©C GHftffiS 

fcmu S&H^-* «:«-3<*»*ao)||tT«:5feffor 30 

^ > h ccmie u xrnmc c Gfi?*f ^aijifctf 5 «t 5 tc-r 
^cttc^-pT. agW£j&B$ec*$<,>-c, mm (!]&£ 

fiS) Sr^A-xtcff ^CiAJf*. jSiU©SI3£-*> rfc/t 
[00 18] COSiSBrtt* S/c. jSH^-^CDCGiij 

[0019] co^B^ru. mm?-i><Dc GWto&m 
user. a»**'? t JHWB«{tt««»«-r«;TA.tt:tt so 
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[0020] 

[KH®££t®Jg8] «T. SM?:#B.§L-o-o % C©& 
W©^JS0J%»^-r5. C©ffeWK:*iC>Ttt t ft 

[0 0 2 1 ] Bl£#l8(-f&£. CCteJ*. C©|fe!J§© 

U : central processing unit ) I. Hsitii&Mjl *:V 
(ROM : read only memory) 2. 5>i^A7i'-bXy 
*D (RAM : random access memory) 3. A^J^IE 
4. *t-SI5§etg§Sg5> A^-f>*-7;*-X (I/F) 

6. atjft&asa 8**11*.. cti&o«e 

[0022]IltCtel>t, ROM2(C«. CCDisXf- 

BB-rS^a^Aai^ft*., CPUlli, CVIsXt- 
R O M 2 «C IBte 3 tlX t > 49f 3£© ^pmK 

WM*y»-A*tt*tMB««:JWf-rS. RAM3 

», cne.©*iJSSJccKt,r!^^7 :r -*-?>^7^-^* 

[002 3] *-#-F. 

tetiLmAt)&re**:m:ffiLxis o % tMW*** - * 

t Xtit z tctb<D&mw&/&n* - . y > jKT » t-/ 

(4g) *mif^5&«c -fe h L/c 0 H|e$ -ti/c ») iEiigic 
«!/§3iii/c«>©Sa7T>^-> a !^BJtC» 
^*5«^^^n5. C©A^S4tC«, 
S£*©miFIII§^-»-fe-9- -(1f«(Dttfi)i^t!S6l3HR 

[0024] ^suieti^s 5 «. mtit&m?- * %.vc 
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C0025] A^-f>$-7*-^6B. nm>mw* 
wmfrtzmmmm?- z ztcitxD j > $ - y x - 

*-C*<3. W*tf. ^g|J©M I D ItftfKn*>6M I D I 
*if-3<«:SW4M I D IA*-f>$-7i-Xtt 
5Ct*!-C*5. ^©A^n'^-^x-.xtcU:. t© 

*. awmflubsfta^ttf 5 -**. m«M i d i y 

F^©BfSOr-i'7t-7? Kc^L-tei 10 
[0026]f«KI7B. ^U9^btM?n5 

i o ». wbs *ifc**« T±n tmiwA**vtym 
#fi#««s^s l oacfxtr-a l l at. Ft/* 

[0027] a^a«g8(i v 'iZ9*ftb-CW»ffl 20 

ji!ll^I8Mf-f^7'l"f 12B, a^WrADP 

SM6£6fctfSC fete, HftMfllHIHR® 

ffi*^©fSM«li«Hi;RO s C ft tcW 5 i*^*7F«: ot, > x 

»«:^JW^-r4Ci7&it?#*„ 30 
[0028] 02SCW:. C©^Hj©-||tSC»I(cJ:4^ 

CO 029] Jz-ir^i/a-AStt. iH^T^sSI 
mm&tZbVT'&r), <fc«3 JMfrffjfCtt. MI DIM 

[0 0 30] C©i^— Jr^-tiJa-AStCB, MID 
I mmw> tomtom?-* *«BIl/T««*s?a- 
^tc^^mis tm 1 d 1 x>^>j SriJi a, £-*•©$ 
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7-"- 5r4fig*i>a.-;U^:^— y>lJ-*^jL-jUS <bL.r 
[0 03 1 ] ^iS^^a-JbAtt. i/-<50U-*i^- 

(wsJa-At*!). 86*0*^*0. eabiRiiaaB. 

[0 032] BjB j e^a-Jl' I «. H^fiX-t- FCCte 
t»-C. >tMri^ -A, S SXtfMMfttt 

X T A D P © f 4 * y V A SIB ± (CSS 0 # D © «t 5 tt 3 

%7cisat4-^x» f u ao^©abfp*f(jfp-r s 
fls+^y** f D©sap©tbfp5&jKX*iiSii/, 

y*d»hDCC. #jB*y»~Jl'AK:J:***ftja©aitf 

ccisiwuflos&^s^ > - *©s:;£K:j&C7cfti;Dj^© 

»f|p *« 1 3 * •& C t * £ . 
[0033]I3(CB, B*&fre-FtC*l>-C?w X 

nrfeij. c©F«r«. >>r>f ^tMD&tF-A©;-* 

-yi'ir^^BD 1 . BD2*33C5EttB«*yyxir H 
tl/rffl«,»-C^«. JJTF. MI D Ij**^-***^^ 

nS*#*HfflItt«?:*'JfflUT. 1>-^>Fi'X^A{C<fc0 
SIS 3 n ^^©ittTtC^te-ti T C *l 6 © # >-9-M D . 
BD1. BD2?:SS6H±S0lJ{Cour. 3 6 {C^«£fl«3(C 

[0 0 3 4] C©iii^^x-;H «, mi&£M*- F 
*y»-ibDMtfli***«. C©J:5iifi»3*©S^IiS 
jw*. 6iT r y>i«Ke i e- Fj tP¥A/s-7J<-$i8:S 

cn?>©!S0#© — 5T*^-f>^>tMD4 
WK:L/fc04K:^3n-SJ:^K l S^>tfMD. BD 
1. BD2«. OD (ft) SPEL. 5r (§5) SPAR. 

SgPLG©?f. iH. ±*», *«r. *^©^a»spJligP 

IH*«. siafciect:. 3 etc. w. JR. 
r c ti h * olSftSP <!: r 4 C i AST * 5 . 
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[0035] C *K«#JI1) H5fcti. HUB* 
[00 36] ^>tMSS*- KtCfcl>r«. 05tC?n3 

ftSi^w:. *y>if»{c. ^n»?Dsi i(cr^> 

1^t»lilf r -*i*Baf 1 fW, 7'n?i'DS2K-C*> 
*©#TMas©»fta*S*aa?U 7*n »^DS3 tec io 
^HIMlPBSfcKSBl^- * ^ + > * JH*saHMJ©*7< 
7^-?4S^-rS. £©«-?(;£. £tc, mmtitcaj 
I&aikD9$RS|5ARK:-3<,»-ct*. 7" a ? ? D S 1 2 V9b 
fE&R&l/7P ??DS4 CCT'hB*tt©ttfFMB£l* 
ffltcRSl/. HfRCC. JE»LGKoi,»-c*>. T'n^D 
S 1 3 "C»fl5*«je L/^a 7^DS5 K-C/MMM!fc©» 
fWWfcBHKtt&rSCiJPrfrS. ft*. BtSBEL 

«>. H«©I$fflS5£4fr *> J: 5 tc^i+-r-5 C <b 

20 

[0037] igeta*. |5©^o-^ds i i , ds 

12. DS 1 3*d«*IK^O»f DS l«C»iKr4 

tfiSSti. I8tcU. ?*DS3CC2tJt;-rs r^r- 

DS4cc*n&r* rj»©»fKjsj ^wrnyuffi^ 

[003 8] ST. A*»B4t«il/TC©J'^A 30 
afBBB*«?v x7uw l 2±Cc«jnSft. 05©S57 

[0039] 06Of-/7O^iilK:fel,>-C, 03©^ 
-T>y>-9-MD. -K;,#y>ifBDlReXA", 

BD2(c>i*)cc»r-s r#>-? 1 j . ry>-y-2 j so* 

3 J ©SfflID 1 . D2. D 3tC«. -t*l-e*l. 7a 

v^dsi ncjttjcors^-y-tsiiif-iitoraa 

D S 1 2 CC#£ 0TK©ttfW>fc6»*tt*Rje-r Sfc 40 
£»© rW©BWtS3e j >A B . 7"D •> £ D S 1 3 Ct 

3ti£Lr tr- htc&fcttfcaoxT-y Twitter* 
fc«>© ra©»fWB£j #*>LB**as*sft4. * 

/c. S^-**^ *7u-fIBffl±<<:lfcLHJ-*-*»5*»* 

{Hsow{ciasa*-r-5/c&© rn^j 

DC. Rtf. &#>tf£@i£l!M1^ttSa>SA>£tISiJW 
(CSffi%Kjr$'*ft:«>© rpKj fx-^^'^XTC 

fc. fefflRti-etucaweft*. ft**, rstgj 
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cc (^^^©fcii^^lHl-jrt^Jr^cc) ftflt^ 
[0040] -f > h a/ftSigtt£3£jK8 1 R tt. O K 

-rs/cj*©^>©-c*5„ HSTawcB. mtJ*7*-z 

[0 04 1 ] CflJftf-fjW^BBe**) H6© 

i j«di© rjn^^-^jsiRj ik»:/db 

Hffij&Jf* -/ * 7 U -f 1 2 fc^n 3ft. C ©Bffi©$B!j© 
T(C. ^ > y>^MD©tftfc»«t*M I D I jR* 

[0 042] H7(C(2. rRSffiaj >Jf*>RB©»ffi 

>t©oc (») . $1? OR) . a. n. ±4*. #• 
ir. ^*©sni«bSiJ©a^©«if^5B*s5ij^sn. cn 
€>©iif^gK:*ti£;L/T-. rtshj ^^^SBSofS 

i*. E7©i5«C. r&-**<t-7imDT. + > 
*;i/jffi8CH. re- htB^j fflBO. r^j «gR ' 
t. r^-juj jasc. r*^ h*7j tiacoccs^ 

^jS-rsci*sr#.5. ft*J. VttB&m/Ujt-tHcr) 
i-»-c«. ^{cjsor. frS©fta«c?tic;L,fc^>-!>-© 
S*Wa4f^^->{c{^o-C#iJEbSI5©7 : 7 *ju f-ss 

ttsicc. aaft8HH*«*»i»r. c©J:^j:f7*jv 

[0043] mi tzifiZtizwwitseM'^ s -tt<D—m 

raoi; (fttf) j . rt^t (fltf) j . -. rn 
(2E^7?I»J) j . rn j it>9 1 6©ttfPl 

D I S^-^C^ + ^^JH CH~1 6CH*5^g 

ssn. iBm©«jfPiBK:oi,>rfc. i&g-r^tMiD 

I jft^r-*©?*-**^ 7'Vda* Ty- h^->j f t - 
frtcRSSti. Sfc. r^j iva*i "6" (c. r* 
^r-;Vj ffl!Vs*s "l.oooo" tc. r*-> fiiVc 
*i "6" tc|S:S?n-CtJ«3. r^-HH^jj fVbM 

[0 04 4] COJr^ftfiJJBSS^^^-df^SIEL/r 

^©iftfp-'^x-^^ffs^ti. se>cc. tt^gffli 

MT©»ttfp«B*^|6-i-* r-fey hj #*>SB©£ 
V ?i>l,c£t)1Z7jkT?>. Wiltf. ttffflg r^oc (fl 
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fpam r^c>D mm j ccot>r* + >*;u#-stcn 

> h o - a j aiRreesuc s&o* r tr- h * y sir 

ft, fficox y t«ck. r tr- h nj^fgj ta^^spB 
R, rebf^^jiij Rje$*»RR fc rthftzr-fri 

Rje^nSBSR* ra* h^^j R5ea5«BCR##R 
[0045] @8(D rf*- jraiRj ^-{Tn^iBffiCCfc 

xUTDA©«iBe«NS. CS, BSOfiHa^— o* 

fRTS. r>>- RSSRNSRO* rn> hn- 

^ J «RK£*C S BTftW^JtS-r^* f"- * £ -f 

•^vdiit, mi d immr-ztp&zj^is h i v 

6t:^7^3nfc r C 13 Modulation , r C 5 D 
R>rtament/TimeJ , -\ r (94) Effect 4DepthJ COO 

U Cht-f^>h I viltR)&r4Ci#r*4. 
ft**, r-/- h*>j K3£SNSCCOC»r4>. Ty^h 

[0046]J/c, rtJ-hir-f SiKRSSPB S 
IT- Hlfifc<y9»J % r 1 tr-h#fifc<7*:/ 

>j > r2e- h^fi<^^>>j , r2/h»*L 

[0 04 7] r^+>*^JWlj HJExyTCAtt, 1 
6©**>*;l/CH 1-CH 1 6<D**>6* Jbft&fSc 
S*W + >* C n *tt*Kia £ fc&<D 

n« % r^-**^ 7j K£:l'J7DA©»s»ns, 

[0 048] r^-^^j S£X'J7DA(D f"fcf- 
h*^:7*J atRH3e«BS©*«tcRW6tlft: T^-h 

<8Vb£ re- Fta^fifij iit "o M - -1 27" 
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cril^I&SSftfce- FtH^fliVbte, eofflfRR^ 
SBBSCD rt-h jr-Y^j f ? — £ B t SiSJRRS L/cti 

[0 049 ] r^-^ii^J y^7P^@iffi<DTg|5tCia 

wen* rsjjf^^fiij Rs*5*»RRMt fc pnMuttn 
rsttftasstt (^p^^-a**i> va% "o m - 

"12 7" ( 7 tf * h ) (D«BrKS-r«te«>©«^« 

>1fMDROV^^^^>-9"BD 1 . BD2 (H3)CDS 
eim%WmftZlr-Jl>$:, ««ffl^r " 1.0000" t L/cfg 

20 [0 05 0]H8O«S«t?tt, r^-^^^yj^ 
r^i->*jI/JBS?J RSxyrDA. CAT*S^$ti/c 

of) j ccot,>r, -<^> hi viu ry- h^->j ^ 

-**a«»3et/^ + ^*JU##Cn«r TCH 1 j tcjl 

BRotr- hm^fiivbo "127" a. r tr— h^>f 

30 SRRR. SR. CRfclt Sbff«^ffilVa« M 6 M (C, 

ffiV s «fS?tffl "1.0000" CC X ^7 ^ h^^filV c IJ-Ko 
( "0" ) CC. *n*hRfi8ftfcttB#*3tiTlr> 

[0 05 1 ] roKj r* + >-fe;uj 

^^y^mt, S7(D r*. + >*;naflej ^^7 

oyiiiffltcR5*s, Rffi*»7 0r TOKJ ^^r>€:fg 
40 K:J:0R3EaM*LttCi*i-&tt. 5cCO*73»3^A^y- 

[0052] r^>^i j <d rsw^-dra&j ccga-r 

&^«W^9y--*OR3e«C»tt«BI*l*AT, S7CD 

r^ + >*jUR«j ^>f7n^iifficD roKj 

1 j so? r^>-^2 j ccociTt, raacD^JBectieo 
50 r % rjgis^-^jiiRj ccnrsiftfp^^>-4r<DR5E 
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[005 3] S6CCfcC^r, r y-fe » h J 
St^ji^ y * ^Tii±ttffJBa{c»j£r*tt^^7 
x - * a*tn j»r js / s 7 y - * cc twraae sti. r y * * 

&8SfP a*flBBR5£./* ^ y - # tCfflilSfgS 3 ft 
&o stfc. yrj #*>*4r^;i/^y v9Thb± 
tbt&sm cc*f jcr a 7 > - # p 7bM*t»E <l 
r^yrj #r*>£*y v^\jidk r-bv hj *f 
*>**y rt^j **>cc»a:-rittf¥ 

SB «c*tj£T S ■frfM 5 ^ - * *s-tfn 71S*RS t te 
[0 0 5 4 3 CM»SCXjeSK>ttfW£#NI) 0 6 (D 

>1M J ttBD 1© rR©«jffiR3Ej stfiOAB&*y » 
PlTZb* HQtc^Sft* TMcDIftffSSj V47U# 
EiB#r 4 1 2*CSI5*3*v C©HM©tgffiJ© 

TCC, ^^>^>-y-MDCDffaSI5AR (04) ©ttfp*£ 

[0 05 5 1 HQ© rRoflfiiRjgj mruzfrnmic 
teor, #:✓-*© 5 r (M) ©sfcraa, ase»*ttccBi 
r2E«»iJ»^j t ra*»««!i j , raa^ 
*t#lj iCOJcSWlSCC^ltfcft. rnoUf^R 

ffla«AT(C5U*3ftrc»*. 3SaffiBATO*fflijCC 
tt. Cft^<DS^I^CKT^>BE^DK^f / P^:S±^f^D8 
0(D/M5^ffi "01" -n. "08" ®Kta^LS|7jrr£/c#>© 

R3E«^xyTAA3>*RW6ftr*3?), fie-^r, 

C© r»©BifpR3Ej fc:^ RBBAR©*©S#«;tt 
MH¥*Rj£LTfcJ:l>*i, KaJAR«c|H«r4CM; 

OTffltcRje-r * <fc 9 «c i/c 4 J: i». 
[0056] rROttffRSj «B«ATKJW&3ftfc 

© # 7 o ^TBffifcffl ir >T K5£ $ ft fc 5 ^ - * cc«5fe 
g£r>T, r^glJB&f^J aD«^y->il/TR3e 

*ft^a:, #>^>i/*- Kcc*5^rM i d i mm?"- 

R**tftWfic»flps-&sfca)«:RS3ftS- 

re*aw*i j ai«R3esft«t. y>^>y*-F-c 

It, *»»&**©*«»*. £#©tt«fc»Lrl<fcf» 

aosaftwcca&^s^^ft, t&$mm 1 j aros 

ftfc«£CCUt» £(DWi, #©*OTK»l/Tl»t*a 

ofitttwcc«ffs*6ft4. *or, ra^ajtaji j 
W«cttfls3*&ft, r£#js«*i j £© 
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ft£o 

[ 0 0 5 7 ] h 9 K5*3ftfc«ro: % rRonflnase j 

*S%iUTAArt© r . j EP«c<fc9*7SS*rC(r>&. c 
©l5^?:l*77b^5Sg5!tr rOKjXttr*t>feJl/j 

y v ^tsatc© ry>-y-RSj ^rn^ 

ffiS (H6 ) CCRS. llMf CCLT\ ffi©^>-yB D 1 . 
10 ?> e 

[0058] acecH6© ry>-^R3£j ^-/rayii 
ccfciir, r^>-y*i j i© rja©iwpR- 

#dr>LBt^y smt. hi occ7K$ft^ 

rj£©s&f^s:Sj ^^rp^nss^^^^u^ l 2tc 
*in3ft, C©BM©^a&©TCc, ^>#>-y-MD© 
SSPLG (H4) ©ttffi*. ^7*^lfm, tr-h 

[0 05 9 ] hi 0© ra©«fffS3Ej ^r^rnyiii® 
cc»w6ft. rj^©«ifiFl9:^j «a«LTCc5i»sfir 

3BB«LT©a««:«. rM©jM¥R3Ej ©ig^ 
ipiaftc, Cft6©J£©IA#^^:±tf©8o©/J> k gi5^{4 
"oi" ~ "os" ffCcRSLS^TS/ca&©R3e*^xy 

©ttf^t, 8/ha5©aoaL«cao % §6©«imc^W 
[0060] rja©jjftRffij aaaLTccjijfssftfc 

JE©tt*«:Stj£urRffi3ftft:^5y-dr4, ^(3:0, 
30 07. 8©ir-/ro*TBan*«c>rRffi3ftfc/<^- 

a-r-^iittj *s^5y-*ii/rR3E3ft*t. 

s«6ft**j, r^f-^j - rjEjg^j ^ b'- h 
^ 5 6 ft fcs©g[i $ zwtm-r z oxcm c ft s . 
[0 06 1 ] fcr-Kcet^s©i6mcot^T^ rax 

^¥*«f*. ^f-^'j 7&^^3ftfciS^CtiZE^ 

r*r*y*j ©«^tt*jB**^sitoiiju % 

40 r^*!? ©Jg^{*ffi££;£^st9mu ra^Uj 
r «^ i l , j r i 

6«^**»C*rR«J, rfflr^y y^EEj *SR£3ft 

SJ ©W^CCttJEfii^BJ^l^tt^rR-S. 3 6>CC, 
rf^^f-^7*^J£j *SR£3ftSi. *JS*fc»3'v¥ 
*ttc>rR0, ra^^y^gj ^s^3ft/cii^ 

50 rf#£«j£JSj 
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C0 0 62] mi OteTnStifcflir-B. rjsooabfps: 

MIDI StUf"- jrCCttttStf-S rg?^- tc 
*5CiA5> S:3£*Sl'J7L AJ*|© r • j EIKC«fc9* 

wsnn,»5. c<D&mzi&77bmmv$.Lx roKj x 

B T+i»>-fe;l/j „ $>-f-3£7c© f 10 

IftJEj # (m6) {CMS. ffe 

©^tBDl, BD2Ct-?C>-C4>. rjS<D3bffiS5£j 

[006 3] J£t±©J:9«:i/r. jSISSfta^tlgftK: 

^^TSCtCCtO. iSSStlfc— mtDJ^jt-* 
*lft**»ltt»5 «:« Or. 7 7 -f;K ftttiBttKB 5 ) 
KfiHWitim*. C©J:9K. #>-<J-©tScB^ 

-f>*>fMD, A-^^tBDl, BD23?. 1*S 20 
ARtWEU **©#>-*©f*©&SI5 (17) tCfllJJiJ 

It. Il©fi. AS. till 21*. S^tCDM'^^ 

[0064] iwm&jsseM&m'} mm*v> * -;u i © 
ju I ©#:^>> y*- Rtfc^r, 

JUS*>6. MIDI tt«9 f -*0£9&X : fftffPmfl. 

(I-5f^©|S]ffl-fi-^ 3 &SW\ Re3t»te'<9-»»-*Ka;i; 
t. 7w*:/u-n 2tc^Sft£#>-tf-©&'5Tib§l!© 
■iff*. *^WE^©*fW»tfK^l/TjE*Wfc 
•WIT*. 

[0065] HI 1 2KB. ^>^^a-;UDM(Cj;S 
^•r-^i&5l7P-SM*^S^-C^-5. C©SSgt^ 

3*1. 9*-jrjr-f^Vd©aBR»3e<H7 r^-ar^^ 
7j flDT) KHL. -Y'Oh I vEP^-h 

B3>ho-^ (c*t jcv-r -s sbfE'* 3 n-t 

OSJfl^ (l38f85£S|5NS. CS) tc3Sffl3*i*. 

T. C©«Jffl:7n-*SSB. -f^>Hff8 (M I D I S 

is? 5 -*) *^{rr siiaaistts. -ec-c. 5ti\ & 

[0 066] C^f-v^SMlJ JHtLfcMI D 1 jR* 
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lr&fflStlfclltoSliCC-^T. HS£3*l/c-»<5.rf-£ 

s„ cc-c. ^>nv*iat3nn>4(YES) 

£*?■•;> 7*SM3{CjI&. I- I v #18523 iVCU 

&t> (NO) IS^tCtt^-f-v^SMl OtCjltf. 
[0067] (Xf !>7"SM3) Xf«^SM3tB, 
3SKE©/MB*8"C(ft#U ^©&9£. 8o©/MB»ffi 

(H9 "oi" - "os" #j«n ic-o^xm&wmmiL 

(beatnow) Nm (Nm : 0~7) *3lto-$"ffi£ 0"CS 

SM 3 X-nm 3 nA;*aE^»#ttNmlc*jC»r«|Wttf|F 
©89^34irt»&l»#£'S#*WJ-r*. CCt. *t 
fWUlff*i|BEStiri»tH.» (YES) i^t»«M5 
CCil*. S*IS;«lff*JS5E3nri»5 (NO) Jf^KB* 

[0068] (Xfv7"SM5) ^T v 77'SM5-r?«, 
3 6> tc. ^KBl«bSB«:^ 3 titers a - ^jiUSfiM I 
D laWf-jr ©-f-Oh I v&-HR-*-£*>£9*>£M 

wra. ccr, — straetiPKBStiA: (yes) 

*S^«^?-f T'SMeKJl*. — St#lt88S*l&»,» (N 
O) ti^fCEiX^-yT'SM 1 OfCjttf. 
[0069] (Xf -^SM6) Xr^SM6-Ctt. ' 

x^5>7"sm i v&mutcM i d nnnf : -f©-^u-> 

CT. tftffi^v h*7ffiVcJy_h©JI£ (YES) tt^ 
7 : '--'^SM7Kji*. 9 h*7ffiV c*SI© (N 

O) Jfi^tCB^f-y^SM 1 OCCjitf. 
[0070] C^f v7"SM7 ) Xf ^SM7m 

rSJS^-fffiJ Vmx raSffXdr-;HBj Vs= r«j 
ffjgigfflj Am 

[0 07 1 3 StiJ. C©^f-2r^SM7©J:5(c. ^ti 

7-cb. jjcoi^Ktt^Kawffta (Pm) tc^ias 

@ (Pm) KiSl*»-arllBB*La:*J60f3e*^3>yJ!Sl 
rt-c^SftS-ttrfeJcii. corns. fflP^*. ■5JlbSl5*ig 
^aa (Pm> (cs«-ra*Tpntt»ci«:75* r *tt 

-rs©*sw*Lt^ 

[007 2] (Xf v7"SM8 ] Xf 9^SM8T?». 
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•r*. cc-c, mmiW&tMsnx^z (yes) *■ 
^f"; 7"S M9 Kit*, tfifcttffawjesti-c^tt 

C» (NO) J#£{Ctt*7-?7 - SMl OJCjltf. 
[0073] (Xf-^SM9]Xf^SM9?tt, 

riHlt^-fflj Vmx rati^^^r-jHBj Vs= TR) 
■f8K*H8j Am 10 

& c oabfffiMfa a max imtssnmtwmtc ( bp 

COSKH&BPo' (c«mS«. 

CC-Cfc. *J-»7SM7&|H»K:, gift 
ffigPo' 41«figiLt. 3H5tfi[g*>*>e©@tIfi 

[0074] (Xfy^SMl 0) XfjT'SMl 0"C 
«. «fflaf-CftC>3a»5©nJ«M»«coi»-C. gfiMI D I 20 

•s^xsf^^^sa^. »^T4'5nb^»s (yes) t§ 

£*-&«mb*#frl> (NO) Jft&tCt*. &©M I D I ft 

[0075] c fflWttfwoM***- 7 a -m ] 

4«:Sf'PL , r > MI D I 7r^;U«r->^7-A{ca-K 30 

ft«ci6Dr»it:r «— aw>»ff-''« 5> - *jw r a m 3 ± 

otc^stir^stfctjcDfco-ra&siUT, «t. 

mil>£'>. ft*. C©*l£, r^flj ~ 

3 J tt. 1216© fi^^MDWx? 

Jfiffir'** 3 rgHEj 

T@«gteai*3itT?n&c^ 40 

[0076] H>i%ls&l4ftft3MH(CS'3r>'C. MIDI 

MI D I SI#f s -df*iSJ^6*l, Xf •yT'SMia 
»>. MI D ISWH 8 -* ©^t>*ACH lilSI 

— ©^-f >*->V#-*ICn = CH 1 **a5&5*VT:i»&»JIb 

gutter, r 1 j © rzEoc (flif) j *mm2 
tiz>. c© r&oo (mi) j tcwM 1 d 1 mm?-* 

© IV- h^->j -f'O h I vtfif-ZZ-i-t&.V&L 
L,XW££ZtlXl>Z<DX. ^©Xf-^SM2-C TYE 
Sj &fflK3*l. X^ >^SM3-C^«E/hgp#{4Nm^ 50 
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StUil/fcf*. X^c;7*SM4{CjlO. 

[0 07 7] (*tf) j kiwi; (W) (cw& 

r swc-^r r&egwiiffj (09) wBBestiwft 

ttm*KS3nrc>&l,>©-C. 7f7^SM4T r y E 
SJ ifflWSft. Xf^7'SM5t (fltf) J 

fcKSSft/e ry-hsM/j vi^iMI D 

\mm?-Z0> rs-h*l>l 1 v £©— fS(# 

mmztiMk. xf-vfsMQtcmts. 

[0 07 8] ^•f-»^"SM6{C*Jl>r. SffM I Dli 

#7*— *©to^"f-^— fit (c cr-i*. 

ftOtflW) Vm#fS3£:*7 » h +7«fiV c = "1.000 
0" JcO^tHii^OiB^tCB. TYESJ i*ij»r3 

n. 3c©xy»ysM7{c4ii>r. sjiafflB (c©wc 

«. 013 (a) (COT<*ti*J:9(c. tJlMUm) *»6V 
mXVs =Vmxi.00O0= VmCO^c^^Jcta^-TftftK 
AitttilCf&ftfcSnttBPoK rax>t;j tfS&Sh 
■5. fi£-oT. C© TSeoOj tt. H L 3 ( a ) (tinf J: 

[0 07 9] mriE©J:5tc r^oc (ftlf) J (cttftfo 

oj tmmsntcm, xt ^smiot. smmi d 

Ilgf-iO-^Oh I VTliif'P3-ti-S'H«BlKjg|5*i 

[0080] C*f«P»fPB#©?S#T t -5r^S7 a-^D 
CCT, fiiC rissosi^Si^j <H9)-C«jttf. 

<bLr. »^smi-c r^^-c (SI) j tfajfbUt 

OJ iflJESrStlTXT^^SM 1 O^UT^f^S 
Ml KIM'). Wflffl©ffll*«fr6l*fl-S*l4. fieo 

[0 08 1 ] U*>L/^6, Xf»^SMlt*K 

(«) J ^BjaftSPiL-T^msmci^K. 
7'SM4(C*Jli>r r YE S J tm«iStixz.?v7SM 

r. 9er r«^r («) j "Am" aaswtmc 

^Kl3-&P)4lft. r^^-c (^) J tt. Xfj 

7'SM8 %igfctt©^ f - :» -7 S M 9 OCfct, > "C, ra^t? 

<») j <z.uw>r*<£itim°m8iit or. a^fig "-a 
£. se-^t, cn?.© r^^-cj rtEtx-i «. is 

1 3 (b) CC^-Ti^K. -eft^n. ttfffl "Am" . 
"-Am" «-«W*ttl/fcHl>K««itt@WfflBP o' 

Ai5*>**IBr-r S*©^'f-» 7"SM1 OtCjitf,, 
[0 08 2] CO^tCLT. SMMID HSgf 1 -^ 
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<da^> h«cjt«^^# r^>^ i j ~ r^>-^3 j © 
oimw±x<Dfimi)mT~?z>k. #©m i d i 

-*©iij*£f5£. M 1 D I VI#f"-$©Sffi®C S 

r. i 2±{c*7K$tis Tjty-fi j ~ 

r#>-*3j tt, MID HS^^-^JCfc-S^ftfH^© 
Mtf tC£fc#T SIS C £ #T * £ . 

[008 3] Ctr-hteffl^J®] Hi 4tc«. 

rUff^ia. ?-**4-?V& (BIT DT) ©iMiRsSJg 
fCHU re-h jr-f^j ^-jrBt (H8 re-h jr 
W ^ J SfR&SSBB S ) flMBrfp* * ^ - * #K£ 3 nt 

^JSftSPl*. MIDI H*^- JrK£«XttjR#OjtfTtc 

s. 

[0084] cot- hjaas Sit. MI D IiH^T 5 - 
h S^^tCjroT. MI D I iSm-f- 9 K.J:Z 

mtomm'pmifimi.c&Mztiz. c©j:5%#fl?fig£f£ 

JKLffittJjrF©* secttoTO*. 
[0085] (W^SSl) h ^ ^ 5 > d^i^f 
SrSfTrSi. X-r-^S S 1 tCtfrBOfrMV 
S55p*«8'JL/> WTCHHR-Cft* (YES) U?^ 
SS2(Cii#. *5-C&t> (NO) J^tt^^SS 30 
3(c*tr. 

CXf^fSSZ) Xf^SS 2-Ctt. qittO'NSitt 
n mfc 1 *flO» l/T/NnR*JBff ( " n m+ 1 " — n 

m) L./t±s ^f^f^SS 3(Cjitr. 
Uf^SS3] ^f7^SS3t?tt, tr-f-if-r^ 
(H7BS) ©SWP^^-f 

*fl#£{s© * -f s > yrjesr 5 <fc 9 teas 3 tut pj 

[0086] (^f-^SS4) ^SS4tli. 
&ffl 3 nfc"5JSfrg&©5tfiE© t- hUNttf'O M 5 40 
feS*>**IS! 1/ . "0"r*5(YES) 9 
•rssstcji*, -?-^rr«t(,> (no) Ja^cctt*^ » 
s s 8 teat*. 

C*f-y7'SS5) *?-?^SS5-Ctt. ^fenTttSP© 
tr- h&N t *. BSSShfct;-- b (cagi 
( "Nb" -N t) -TS. 

[0087] CCC. KSSftfcK- h#{iN b itt. 
rt-h^-f^j 7-9B t (i8BS)Oftff;<5> 

-*<L0T. mittf. r i hJMi j fit® 

SSttT I IB "Nb" = 1££9. Tit- 50 
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(7^) J t&jmicb "Nb" = 17*0. 

r2K- \-rna j -c "Nb" = 3, r2tr- 

HH1(7?^) J "Cfc "Nb" = 3 £#-5. RMSCC. 
T3 f - HHfcj 3WRffi3*lTl»S«tett. tf "Nb" 
= 5£<L9. i"4tr-h*ttj ttSWKCtt "Nb" =7 

©e-h£©flr«Ki«'C**©-e. Nbffl«:«^g?3n 
*i>. *fc. r 1 /j^jnaj rz'mmtit.} tcoi> 

Ttt. l/MB#©t- h&n bfOEC-C. "N 
b" =nb-lM"Nb" =2nb-l±fc*. 

[0088] (xf^ss6) ^f^ssem 

g&©/jN&£8-Clifcftlsft:A9*. &fiE©'MMI&Nm 
(beat now) £3lfc>f fili U 

m s nfcJa&/jNS5m{4 n m tc *$ 1 > r ©s:5£# 3 

SSnri»&t,» (YES) iXfv^SS 9tcM*. *f 
IWttfWISBeSti'Ct** (NO) «d(C|»7>7^SS 
1 3 icjttf . 

[0089] CXf^f ySS8) — ^. ^r^SS8 

r«. ^KTraiscot- fskn t «r 1 tmmnvm *n 

t - 1 " (Cjgff ( "Nt-1"-Nt) U/c±. 
7"S S 1 3 ccjttf. 

[0 09 0] (Xf v^SS9) Xf"^SS9t?il 
fcT- Uti^JfflV b*iS!S* » ht7fflVcai?ft5* ! 
S*i*$iJSiJL.. *7ht7fiVc£U:r45 (YES) 

«^»xf-7^ss 1 o«cjt*. fiivc*?as-c*sti^ 

KB^f"? 7"S S 1 3CCjltP. C^f5^SS9B, 
5ts2©fc»©* v ^©■C^SCCIc.DT'&BS-r £ C i 

[009 1 ] (Xf-^SSlO^f^SSlOt? 

rt-na^fiij vbx radf^^-5r-;Hiij vs= rsj 

fMitiflj A s 

*S*5Ci**-Cfi». C©*^. TOMPUt Jf^R 
WriilHI +©IMW«i8*lE!»Ttt«: brfc < ©*«*?* L 

[009 2] {.7-7-v 7SS 1 1 ) Zf-vfSS 1 1 "C 
«. v-7 - SM8t|5l«(C. SKSTttflStcpgfiir&tt 
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j&*S»£3ivCl»4 (YES) m&\tX7-v?SS 1 2tC 

it*. ^ffij^iSjSsnTt^it,. (no) m&iatz 

7-v7S S 1 2fCjtt?. 

[009 3] (Xf-^SSnj^f'j-T'SSlOT 
fc> X-r -;ySM9 tm&K. ffii&btc 

re-m»ij*j Vbx rgb^xer-jHiij vs = ru 
mmmmi a s 

6 C©S&mSi}§filA s #/dWttfaftCCg{iL/cgttrt4g 
Po' ic#ffibU COSWfiSPo' tc^Stt. c© 

xf^ss i oiisiiitc. g^iagpo* £@*fu>*g 
isj£L&#<=>BFr5£* ■< s >mft-c®m-&z c tan? 

[0094] (77--^SS13]Xf-^SS13t 
-r&"Jf&8|5#*S (YES) ^f^SS3K 

sttsOis-r. c^tc^^-rsoisbgBj&i^tv* <n 

O) J§^«:«. #©e- h 5 >^(i-S|©5fl=&ff& 

[009 5] tr- hSaa^o-S SliC©J;Vj:Af f 
■P'SS 1 ~*^*^S S 1 3ip^r>-C^iO-C, C<D 

(as bs) ©am^^-sicc r 1 1- 

Hiffi (2 , '5>) J 3 tlT 1 t- 

bSC«ttrf1p**--.»MiiV s CC^OtcMtclfSJWsm^tiL 

[0096] twm&mmm mi scc». 

a-;Kcj;^^eiS7a-S A# rjgjataJl ( i ) J 
Cflgi 4©^ffltcj:9MitMfT*©*ffitelfcS0 , rfiij 

[0097 ] ftWumSAlt. M I D Ivg^r-*tCJ: 

saMKawtucfiJHWtttiiiAtc ±0 -exists ct*s 

>a^KBH8**TtJ:l»0. tf- 5 > deltas 

orcn«:iaBiii3*tt<"c4J:i.». ±ie^Stea ( 1 ) 
y US A ccfc ?• v •/-C©®a«fe(T©<fc 5 tc 

[009 8] (Xf-'v^SA 1 D WMZJ 5 >y<l^ 3 & 
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Wttf. c ©{forte. ia4©«fc5«c#>ihW£i£:i/fca 

[009 9] T'S A2 ] X^^7*SA2-Cli. 

&m 3 n ■5i«igp©{agffi^ t b r . iiafig ^ tnmtt 

■i©IBI©ffiltLt»fflr*. 

CXf^^SA 3) ^f^SA3tB, ^PlKlSPCC 
iras*ifcttft'<*y WfcttflSOMt V a * m 

t>-C*<a#Kl&8iLu = La/ (aVa) Cat*. ®fi 

^e^iifitcisi^or c©#a^»js§iitL u tci-tzz®. 

[0 10 0] (Xf !>^SA4] ^•^•>7'S A4-CJJ. 

ttSBtfifc-S (YES) Xf^SAllClO, 
iS^-T S pJStSPtCOCi-C^-?- 7?"SA1 feTF©*fiffi«:tS 

osi-r. 35 fc. cftKBeaTSTfTOawtto (no)» 

&att, ^©Sdii*<i^©s{t*($*s-rsaw©^itc 

[0101] C©?«^AKlSA«:ffiWC|ftHJ-r-&fc* 
K. 013 (c) tC«. «SMIISA3&i||?T3n*^>' 

© rfi^t («) j (j. fi]Wtt§#-^iftim-c^sn-5 
13 (c) ©iEi^tr-s-r^fiiigccifeSi*, 

SAlt? r^^T (ft) J ^plttSPiL-CftlffiStl. ^ 

f»?SA2r r^^r («) j ©Jia(4Si1w^fiig 

6©H©ffiStL*^ffi3tls Xf-'^SAaKts^T. 
T2e^-C <«) J OttfP<9>-*OrtO«f|5ijS>WV 

zfim^hti (H7 r^j «jRT©«i "6" ) . mm 

LfcCCDttfBKSHBV a = 6T^ltUft:fiiL/V a = L 

/B#jjw>6ft. r^^-c <«) j a. -&&m<omm 

&S©7?ia]{cm&^Sb8iittL u = La/6 atSttKtiLb 

[0 1 0 2 ] gEtca^/tJ^K:. iRf^-fAimSMR 
O'tT-hfeliS SCCfcW^^ry 7"SM7. SM9. S 
S 1 0 . S S 1 2 (DH^rmnWr^T -y 7-CK. ffifS 

oJSfcg|5*W^> h I v$>tr- h B t fc.&CXmffl%}-C\t 
^e.c©B^aBic|S]*>- 7 -cfflraboo^tb$-i± > oJKj 
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[0 1 0 3 ] m 1 6tC«. #>X*S?:x-.iMCj:&SIJ© 

j&n&g^n-sAa* rj^jt&g (id j iur^sn 

.fc^tc^ra^s&^S'&tsffl-r-s^cciafflsti. 015 
© r^jaa < 1 > j is^*ic5«. ttm©&us(c 

[0 104] CWf77'SA 10 

r*Sfr§3&>©«itCi;t. «jttf. fflfl|(ti©StoR!ft& 

[0 1 05] C©«fc5CCL-C, 3oO(SISM, SS. 20 
SAICJCoT, #>■?*•©& nJibaS©*bfE£:. jg^lfflfi 

M-v-eov ( I v ) jyess*. ^©t-hmuss 
tt-h (Bt) (cftss-gsor. g&ttMitzmtu 

C0106] s *=>«:. mmmtico^-cte. nmsM. 30 

S SKJ: £**©■*■'<> hRtftf- FfcrSST SUffiffJ 

abf^ «c*t ot, tgm&teiifi&s©^*^ s a * 
*s-csrs. seiics/c, c*i6©siffiKj«&tt©£{fc. o 

SO. Xf^SM?, SM9. SS10. SS12IC 

[0 107]«±. H»*^**hiO-cy>^tffl 

^SJg^©.'"?^-*!^-*- F (^>it^m*- F) 40 

[0 1 08] fllittf. Xf-»^SM7. SM9 . SSI 

0 . s s 1 2K*itrsin»*^xi' h^nttaposcttii 

flr©S*g{i!I <ftft£3tr) «-3t»rB. Xttffrct*. 
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[0109] 4te, f-f^^U-/ BB^©!ftffeilw«:-3 
l>rtt. etaSL/fciBflR*^** F©B« (H6TC< 
BIHffll>) ©?k «*©H»BBE. HflUfiS. MM* 

m©fe. ttJW*©fl*©we*fT*.*J:9fc:U 
±fc. ®®MS{Col,»-C«. fl^L/c*y-7 (*&.£) (i 
H©9tffc. HMfttf^y** F©H$e (06TC<@SkM 
S» ©*. 1 71££K©TMfc***6©»to*&JR 

*s-C#S. 3S>tc. #aWHH«*«©a«>gil*. 

tottrStftStffcO. Wfc«*«:|»UA**«4 (El 

1) (om^te? T>?i'3>*- : £&v?LxA&,ttiicm 
*©wmf**fT9J:9eci/t. —m^r^mmh^ 
n z£ *> {c-r * c <b fe-c* -s. 

[0 110] SW^-droCGHWMBJl^©** 
*BGMLlsfc9. ^fiE3n-2»i!l^i©|Sl«9afeaf«:S-nri6SB ! 

ffi ©is]»i©5illtt*— ®ra± s -&s«fc 5 ic-r -s> c i 

[0 111] c©J:9«feit*»HS*Jtffl7bSe»U 

y * a» f ttmmKmmi,-&mm®i*'£.f8.irz> ct* 

F^^SBSfP^ttrKWD. ry- F*>J 1f*K©/- F 
*->'<©«£*> &X8«ff < ■5J«)S|5©14g* : J^'J-r€ > C 

( r fj . rsj. rvj oJ:9tt) fo$«iWrU c 
^ F* J tT^*?*<<>rt,^'>->-c»n^ -e©^m© 
«t^K-r-s©-cfes,, 

[0112] sfc, mrasL/fcttFaijaatc-^c^Ttt. 

■nmmte&m. (Po) *-c*. f-»j<««. r-y-f 
■ >aue»&iSiiBK««oaii/iMifli^ttd4A:4. a 
c»b. c©s^filg (Po) $-e£t-Fisj*8-c?iP,gu 

*fc-JilS|±T S C <b j&J-C * S . 
[0113] tt*-^-*4>©XSffi*K*l. 0 

$0. M I D I iSS^-fWOBfig r^n^^A^x> 
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fc. C© r^O^Afayi/j tSfR^^tHCj; ->T f 
T^Wfex ;M * 'J ^fe^rt* &£>©"?> C©tt$RK:*t 

©«fc 9&*^#©»MbflMCOI,»-Ctt. 

a. 

[0114] E^f 8 -*©*!!**?*^ WifiLfcAO 
fflJ«:tfo-r*5< CiB. rt/c<5 J tPf«^&^®©S 

nco*iw±Lfco. £flj3ti**ff&©BWMfctftc&«iJ 

t©<fc^5fcii*fl?tf£*T5/<:#K:. C©fS 
?J©#* LV>3lGHdNI(cJ:ti». iSSH^- * ©«£* 

[01151017 leti. c©&93©wa&^iSfl£8l«e 

*«s«T*asi«:te^r«. ?gi#-r-£©^aib 

*D. , Di , Di , -fre&fcSSIlT*-*©^©?! 

£©||£*^>*RP£W:J5iJ®lClS:tt6ft£5t 
ig^-O^PPtt. RP(cj:»)}g^sn 
•SUt^-S^P ? £lC*tl,T. MxB, Bf5t& (n- 

[0 l l 6 ] Sfcl^tf-Y^PPW:. igt^-f^Ssgfta* 
jHR3ft<5i. iH#r-$©#&#f£^*K -5>I5K^SI 
af 8 -**^*^*©**?*!!!**!/. SWrl/telS* 
*iBtH«S«:iatSJ-iit:fT<. 0»Rtf, B#.£t.., Ccr 
5fcK*tf > * P P «c «£ 0 SS^t 1 - * © 5 * * ^ o 
7fD. *ng^3nsi. c©5*-*:7ci ?*D. ©?ft 

tf-iujiMsns. -eu-c. c©iSi#i-f-*a>e>. 

mo». e©^>hRtf*©B*M*«K*rt*£l'"C. 
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<tLri2tt3-t*£. -s-ut:. cosifi^-*©^^ 

«f <t... ) tc. Ett«B*>6IWWS»*RHJb'C»lS: 

T 5 C GPi^?:*^^^ ?■ A D P «Cfi51i$ -t* Z. 
[01171018 «:«. C©,}: 5 &$t8#JB*T*Qi!7 
a-SE©— ^ifefll*«n;**n:te9. #S!#5iW>*P 
PVCDtm (A) RtWflfe#-f RPTCDMfll <B) 
*»6SXS. WfitsKOfRPTOJES (B) It. SJHtt 

io PPWMI(A) fc. jfcMWMHi&WCTriBarsc 
i*Wll>*5. fe©SS&%£ <«*.«. 
*r©JKUI) ©:&!!**»,» £*(cttes&3tit\ sfe*j# 

[0 118] (JfeS^tf -0**3:81 (A) } 5tgS^ft?«T 
*S«:7Ci-SE«Ctel.»-Ct*. 3te?\ fclTOS^f-f "^S 

e i i~se 1 4*^fiS-53ti^**;^>f saa (a) k 
iot, «ffir(c1SIil09MI^3n. *©«. w^-o 

(B) **frbft3. 
[0 119] (Xf-^SEl 1) -f^>hffll4fW 
20 SCiGC^SfeSE^tf-f^^II <A> tfgSfrSftS 

s. Wttf. Si 7(c*ju»r. 8#^t.., cc-c5feijS** 

-f>^PP(C<J:0^Sn-5f r -$'^'cif ^D. ©jg^ 
[0 120] CXfv7*SE 1 2] *r-s>7"SE 1 2"C 

6. ate. coxf-^SE l 2{cteW5fl?*fCcii. ^ 
^isi^^-^D. office t&Rti. ^-ntiiftc^fsn 

£. 

[0121] (Xf?7"SE13)^f55'SE13-C 

fcC G^- * -f > * 4 *KEtiatHKttlftU 

40 r. ^-^SE 1 4Kiitf. 

-f>5fPP=&lo«Jt)'C. ^©HiA^^1ST^t»3CC 

[0 122] CW£*IW>**yi <B>) CO**** 
fv^SEll-SEl 4*»6«*5feito#-r>*«ia 

(A) ©«CCfft>n*Hflfe!l« (B) tt. «T 
©^f t7^SE 2 1 ~SE 2 5A>6fS2>. 

[0 123] (Xf- ?^SE2 1 J 5te&*5iW 

(A) tCfTOIhSfcot-f^HIHI^SCi 
50 {Cj:0S^-r>5?MII (B) *s|g«i3ti4i. X^-y 
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Ell 7K4sl>T. e^t... (C-CB£*^>*RP&£J: 

ft. ? E2 2JCittf„ 
CXr-?:/SE2 2) ^ry7 , SE2 2?(i 1 tfetH3ft 

[0124] (*7 i v:7 r SE2 3] X7 u ^7'SE23r 
tt. 7*S E2 2(C*JWS«|f«iatcMtf 1/-C. 5fe 10 

SS*B$ C5feR*#'f^if«ia (A) ©^r7 7-SE 1 
2D K£|*»#f s -*^LTW«LTfct»fcJKtftt* 

(CGf-f) S*#-f>*«|R:b"CSalfi6iB36» 
X7-7?SE2 4Katr. 

CX^^^SE24J Xf7^SE24t«, K#fcH3 

U Z.rv7S E2 5CCiStf. *©|g*. -rv^^U-Y 

i 2©siffi±{c« t ^Bantf-- * <wt«. D. ) IC 

CX^^^"SE2 5) X?7^SE2 5-C«. St^-f 20 

o^fMFRum&iMra^ai^ff s ft -c t> < . 

C 0 1 2 5 ] ±.j£OtcfflXlt> ftSi&lBlUn£.Z 'J T)V 

|fr«:. midi 7t -oua>s>©is§#-r-££^**&yi 

[0126] JM_h©«fc C OJBH®ftB*JB*rffilI 30 

i?htUt7^ h^S^T tC*a>T. 
h ^^©njsdgp* try - x h ©£#©#■ <h L r 

[0127] iffirm&m) mftbtexotc. ?g#r-* 

*^f-»^SM7. SM9 . SS10. SS12. SA 
-6. C©^©^J©JfiF$L^HJB^CC«J:*xK > cft6 50 
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©IMWW*;*?- »:/«:** l/T. IS <e-h) ^/hfflKtif 

[0128] BP*. C©ftH®ttR«9tcJ:&&. Ch 

e>©^8&3t^*?-* 7'iC*tbT\ U-A£*iJ 

BuritnMftffltiitfr* rffigB$m5tc*t**ftB8iia 

Kfflia^yasircsft. co^^^fct*). ^pj«j 
ft£. 

[0129] tmmmm&icisv &mr*m$nww=mrn 
mm < i > 3 jtr. rfe@^mft(cit»ts»mEiBtM 

>^-e©s^cGJSiif s ^y**-^ i/-Akf 

i, kf i+l . ••• < i =0. 1. 2. •••) ilTSSL 
r*str. **-7U-AeD^Hfti'j©»iBiia»tW«ir 

co i s o ] h i 9«c«. a«nn»sft«tt ( tr- h> * 

tctiXDZJ h#7F3*iri>£. -3*0. C©a 

Sftfcatlt* -f 3>yb i . bi+i. -(cMl&l/'CifiB 
*-7U-Ak f i . k f i+i . -*se*3ti&. ffiW 
IW^tJ. Ctie>©+-7U-A©Pa©fflP^|5i<^>f-c j 
<j=l. 2. •». n) CC-CnlIl^f3n**s. C©% 
ig©«raH5a$IJW©^Si{C«eo t . fg3£B$PflS ( k f i 
~k f i+2 . k f i+l ~k f i+2 , •••> »C*jW4lSHia 

»n*«i'^-y-A©AaffltE*«cj*:DrjfiFa«:»iiaisn. *t 
a?r*iii*»s*fcowiiisfetT9c t«Jt?**. 
[0131] ei 2 ojcii. c©i 9&MniaikM9P(cif 
»5HawHi7B-fi3a« riines ( i ) j tL/tss 
nri,>s„ c©ffip^iaa ( i ) ©^cj-sNii. 019 

ur©fe©-rs>0. ^•5 u y^"r©MSt3:WT©«J:^{c 

[oi3 2] c^f 7 7'sn 1 ) c<ommn&m < 1 ) «« 

2(Cjitf„ 
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I*?-? ^SN2) ^x^T'SNZ'Cl*. iSHtiHW© 

giJL. teWSfim 5 IstfX'&Z (YES) iXf-^'S 
N8iCjI*K -e^tNSc^ (NO) i©^«X?-77*SN3 
(cittf. 

S#c j =&^«<£S«DfilCC. J ffi^Snr^/cffl^lHllS[n 
iibf.au c j ^n-C&n«^f »?"SN7H:SI#. * 
5"Cttlr> (c j <n) f-^SN 4 Kjitr. 

[0 133] (Xf 7^SN4Uf 77 , SN4tl* 1 10 

yy> l- orfii "c j + r &cm#t ( "c j + r 

-cj) L/c±. Z.T-vZfSUStcmts. 

9bS : B&ft-t^MjHillt) lZWife<mWlVL n =Sr § eUCsll 
S**ffig£-C©*-7 U-AlB*IW*An = K n X V 
AnX (cj/n) = rflmXttBj V J 20 

"C. MB <»i - 1@) fflllltiAfe&COttaccgKnJ 

©Trr«>g|Sfcoc>t|SJ«©Mffi£tT<.>> tttf*l«'X©ilStt 
*fSW*tt»«CR*. Xf-^SN5t. A/ 30 

n= r*fiHMx<UKj vu*$», ^-^snb 
■c. *b <* j - 1 b> *rntt«*'&¥flaini3Mt«v 

u A*tf Kfil 3 *fcffi«*J8Drr S J: 9 (c L-r t «fc 
[0 134] (X7--^SN7 ) *-y-*:7SN7-Ctt % 

'J X > h l/Ttf " r + 1 " CCSff ("r+1"— »r) L> 
. te±> X7- 7 :/SN5«ejttr. 

[0 13 5] [Xf -^SN8 ) ^7 s -^7 , SN8fi3:. 
*-7l — Ak f i ( "kfi+i" — k f i ) 

U x^v7"SN9«:jitj„ 40 
ir^ "0" T*5#>S#>*2pJJ5iJU r = 0"C^i (Y 

es) txtjT'SN i otcii^. -esrfc^ (no : 

r>0) W^f^SNl licjitf. 
CX-f-^^SN 1 0) ^f^SNl 0T[J. fflBH3» 
ntWOH^c j &Cjg«Ff ( "c j" -n> U 
XfvT'SN 1 ZiCMtS. 

7SH 1 1 ] ^f-^SNl 1"C«. ffiF^lHl^ 
n £MMBftX4bB r a 1 *** tifcHn + r JcMSr 
( "n+ r M -n) Ltc±. %=rv 7SN 1 2tcilt?. 50 
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CXf^^'SN 1 2 ] Xfi>7 , SNl 2-Cti, &©*- 
7U-A k f i+1 TCDtoilK-fS^SfcS!). ^KoIWlgPtC 
out. HROtf-f > hfHIc j &CflfflinB&Kfl:&r * 
**fil "0" {Ci7J83{bLfcJL Xf77"SN4~SN6 

Kilt*. 

[oi36] ftfe, ffir B ^ii ( i ) icjzzmmmm 

v^SN8%gil/T^f -^SNIO, SNlirffl 

{*. W«iSI5©^£{ig*^glft{ig*t?©ffiRS!ERB <£ 
^»ja#H) £«&©*-? U-AtC*/£*5^ii-S^355 
lot. Xf^SNSKfetfSMKnttl* 
WDfltC-r«<:&tfft7*l/l>. StSpTSii 
Siltco^-CIESfSn/ctiraiHl^n?:. e(K©t-7U- 

o"C. iS£cDl?»gR4COC>T«&$fcKn% lfe(± ( 1 * 

-7u-A»!ra*ii) £-r-5ci*irts. 

[0137] fcLbO^v^SN 1~SN 1 2#>6JI8? 
SfrSJ^Jc. ( 1 ) (CfcSi, 5fc©»fE 

CI) aig^SSi^-^b-AWrakf i~k f i+i -co 
f i {cjgff^nr^?.^©ttSSf^^ 5>^Bi+i tcm 

STS3ET©ra. (a) ffira^-f>h##c jlPfefflW 
Btt«4K£MHIBIft n T * * rB, ^7"»7'SN2 
~SN6KJ:-?r. C©ffifBJI!&c j /cW*IISJ*fTt>. 

(b) mmmmc j *K£ifraB0tn*tt*.«&. 

>h ( "r + 1 " -»r ) Xf^SN5. S 

[0 138] C2) ^Ot3iii*-7U-ASSrak f i+1 

~k f i+2 icft-rzmjmit 

'XOteWM* A S>i*Bi+l {CiUii-TSi. 
Akf » 7'SN8(cr^©fSia^-7 U-A 

kf i+l KMfSti. JfflflHtfcnK-^TW. (a) K 
jatMBtt n «T<DXMfmBft e j r»W * -f 5 > ^ 
Bi+ltCidaiUfctB^ (r = 0) (Ctt. Xt-jT'SNI 

okjtot. c©s^fflraB^c j ikwc&fmmkn t 

U (b ) ^Sfflra!5|i!cn^ja^Sjll^»rail&n+ r 
rBW3fY3>^Bi+iKiiJitl/te»^ (r>0) CC 
». Xf- v v'S N 1 llci-jt. C©HEg*iP B 1lEllSn + 
r^iaSffi^lilftniL-r. E«C*©1SB*~7 L--A 
kf i+2 ©MHfir©7U-A»!rBlkfi+l~k fi+2 
fl>lilHlttf¥«:fl|jl4-*. > h#-^c j SO'ffl 

FnllsJ^fbS r tCO(,»-C«. Xfy^SN 1 2(C*iC»r 

"0" «Cl7J}W{b$^S. 
[0 1 3 9 ] 
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(a) ?b-AjWHk f i-kfi+i tctei^T, *ISHc 

^titcmrmm^^^rtx^tc^naT ( r 

= 0) <Dt§>^iaZ. ffir$CD£j]&tf;frr>tc±b'C, CCD 

7i>-&v$mutcffim7r^ > h§fc j whmmcn 

(b) 7v-<ummk t i-k f i+i cc*5i>t, fflts 

fWH3EEI»n «:iB*.5 ( r > 0 ) <D»£GCtt, R£S 

**»©tMgKc*tL>r, a«-*"s»iii*-7 u-a*»6 

*fc. C<D«M«H<1> ^ 

[0141] C*BU(Rd«:J:*»nn«IW=flBI«a 
(2) ) *cc. rii»|BJHl^«:J:iliBI«»j I*. SfW" 

«it«je (br- b ) *>/M5*>* 9 *s&& 

5>^Sr*-7U-Akf i . kfi+i, - 
<i=0. 1. 2, »•> ib-CS^T^i^CC, 

icfRwaraieiBfiD«:ass-a:r*j*. xttSisoBaM 

*>6<oeiftl»BI tm*CCE)Hte«FiaiTkf4JSiB»CClH^ 
* - V U - A CC^iJjti 0 /c 6 *X©B*|I8S d rt x~<offim 
[0 142] 0 2 1 tctt. c©J:5«cB*iafla-&«:J:4a 

mtmn^fT s * /e #>cd * a ^ * - h ^ 

eauojaa^ a -ww rnwiteJi (2) j tbxnk^tx 

[0 143] (Xf77 , Sll)C(DiWi(2)^ 
^^f-A^at^CCJ^GTSSSn^^cD^PHlPHlPS 
O«ii»ecJ:0filbSh, 1*PBl*-C*)SC<b«:^rfflPBl 

WHSIaii. ^f^S I ltctet>T, ^t£°J 
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-7U-Ak f i+ipHteB$fHTlcf£J«£U ffiftRHI 
t m***-7 U~AM^mTkflcHjil,Tl>£ (YE 
S : t mSTkf) S^f^'S I 5 Kit*. «r 
(NO : tm<Tkf) T'S I 3 ICS 

to. 

[0 144] (X7**?'S I 3) Af-yZfS I 3tB, 
SSW«©^o2/f- y HSV i8&^X^r-;HBV s com 

"C 

A i x { (Tkf- t m) /D} = raHH^ffcBj Vm 
Vm^«) 4 ^f-v^S I 4fciitf. Sc*J. «K i 
U-A)0IH*#^---r5J:5«:, l*«iT5CiAi» 
20 C*^*^S I 4) *^*:/S I 4t?tt, HMttl^6 

r, snaofirattBJfe&ccDtt^ 
± v y*-:>u fflpa^^y^iErenri^-XcDpJW) 

[0 14 5] [Xf I 5) «>^S I5ttt, 

t-7l/-Ak f i £H»r ( "k f i+i " — k f i ) L 
X % fflfttiffST kfZWM ( "Tkf+D" —Tkf) UX© 
30 *-^U-Ak f i+l r©BB>6&B**«>fc±. 
«I3. SI 4CCiltfo 

[0 146] tUnD^^'S I 1-S I 5^6S»S 
n^<fc^tC. CCDfflPa^m (2) CCcfc^i, VXfA© 

e$p^ tmtcse-pr, *©i*©*-7u-Art«pdMifi 
jWg38«:#a&&ft4. 3sfc, c<D?fPa^Il (2) tt, «f 
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Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer.So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
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CLAIMS 



[Claim(s)] 

[Claim 1] It corresponds to a performance data-processing means to output musical-sound 
control information and a synchronizing signal one by one corresponding to the musical piece 
which should be performed, and the aforementioned musical piece which should carry out a 
performance. A parameter supply means to supply the operational parameter for controlling the 
movement of each part of a picture, Based on a musical-sound generation means to generate 
musical sound based on the aforementioned musical-sound control information, the 
aforementioned musical-sound control information, and a synchronizing signal, it has a picture 
generation means to generate the picture by which movement is controlled according to the 
aforementioned operational parameter, by the aforementioned picture generation means The 
musical-sound response picture generative system characterized by generating the picture 
which moves in accordance with advance of the musical-sound generation by the 
aforementioned musical-sound generation means. 

[Claim 2] The musical-sound response picture generation method characterized by to consist of 
the step which generates the picture by which the movement of each part is controlled 
according to the aforementioned operational parameter based on the step which outputs 
musical-sound control information and a synchronizing signal one by one corresponding to the 
musical piece which should be performed, the step which supplies an operational parameter 
corresponding to the musical piece which should be performed, the step which generate musical 
sound based on the aforementioned musical-sound control information, the aforementioned 
musical-sound control information, and a synchronizing signal, and to move a picture according 
to advance of musical-sound generation. 

[Claim 3] Furthermore, the musical-sound response picture generation method according to 
claim 2 characterized by to have the step which predicts and analyzes the information on the 
portion concerned and prepares the graphics data corresponding to the information on the 
portion concerned, and to generate the aforementioned picture corresponding to the information 
on the portion concerned using this graphics data before the information on a portion that 
musical sound and a picture should be generated among the aforementioned musical-sound 
control information in both the steps that generate the aforementioned musical sound and a 
picture is read. 

[Claim 4] The musical-sound response picture generation method according to claim 2 
characterized by carrying out interpolation processing of the movement of each part of the 
aforementioned picture according to the throughput of the system for generating the 
aforementioned picture. 

[Claim 5] The musical-sound response picture generation method according to claim 2 or 3 
characterized by making the aforementioned picture into a musical instrument player, analyzing 
the performance gestalt which this musical instrument player should take based on the 
aforementioned musical-sound control information, and driving the movement of each part of the 
aforementioned picture according to the analyzed performance gestalt 

[Claim 6] It corresponds to a performance data-processing means to output musical-sound 
control information and a synchronizing signal one by one corresponding to the musical piece 
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which should be performed, and the aforementioned musical piece which should carry out a 
performance. A parameter supply means to supply the operational parameter for controlling the 
movement of each part of a picture, Based on a musical-sound generation means to generate 
musical sound based on the aforementioned musical-sound control information, the 
aforementioned musical-sound control information, and a synchronizing signal, it has a picture 
generation means to generate the picture by which movement is controlled according to the 
aforementioned operational parameter, by the aforementioned picture generation means Musical- 
sound response picture generation equipment characterized by generating the picture which 
moves in accordance with advance of the musical-sound generation by the aforementioned 
musical-sound generation means. 

[Claim 7] While making musical-sound control information and a synchronizing signal output one 
by one corresponding to the musical piece which should be performed, supplying an operational 
parameter corresponding to the aforementioned musical piece which should carry out a 
performance and making musical sound generate based on the aforementioned musical-sound 
control information The storage for the musical-sound response picture generation which 
memorized the program for performing the process to which the picture by which the movement 
of each part is controlled according to the aforementioned operational parameter is made to 
generate based on the aforementioned musical-sound control information and a synchronizing 
signal, and the picture was moved according to advance of musical-sound generation. 



[Translation done.] 
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* NOTICES * 

Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. . 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Industrial Application] This invention relates to the system, method, and record medium for 
answering the technology which answers musical sound and generates a picture, and the 
information which interpreted and acquired the musical-sound control information for musical- 
sound generating especially, and generating a graphics dynamic image. 
[0002] 

[Description of the Prior Art] the background to which a picture is changed in connection with 
the musical sound generated secondarily to the main operation of advance of a game although 
that to which a picture is changed with CG (CG:computer graphics) according to music has 
already existed partly as game software — it is visual (BGV), and musical sound and the picture 
are synchronized beforehand and a picture is not finely controlled using musical-sound control 
information etc. Moreover, it is not found, but there is also a psychedelic thing, the layer which 
can be enjoyed in the field of an environmental image is limited, and it was easy to get bored 
with the thing on the theme of the object which moves by such game software dynamically 
musically like a dancer. Furthermore, according to musical piece information like MIDI 
performance information, a light is blinked or what generates a CG picture like an environmental 
image has appeared. 

[0003] There are some which perform the dynamic-image display which was suitable for music 
on the other hand using the picture pattern, without being based on graphics. For example, the 
musical-sound imaging equipment which judges **** of the musical-sound information from an 
electrohone etc. to JP,63-1 70697,A by the **** detecting element, is beginning to read two or 
more picture patterns to it one by one with the selection signal according to this ****, and 
displays a dynamic image like a dance or a geometrical pattern on it according to **** is 
indicated. However, with this conventional technology, since necessary musical-sound 
information will be processed into the selection signal according to **** in a **** detecting 
element, the dynamic picture which suited the musical sound of a basis exactly cannot be 
acquired. 

[0004] moreover, by the method of using picture pattern data like this musical-sound imaging 
equipment Although there is much amount of information, once it must prepare still a lot of 
picture pattern information for obtaining a changeful dynamic image so that musical sound may 
be suited further and is moreover set a top deficient in change from the first Since various 
change could not be arbitrarily added to the picture displayed according to liking of a user etc., it 
was very difficult to satisfy various demands of a user. 
[0005] 

[Problem(s) to be Solved by the Invention] This invention was made in consideration of such a 
trouble, and one purpose of this invention is danced synchronizing with the performance of music 
like a MIDI musical piece, moves picture objects, such as a hand (dancer), and is to offer this, 
the CG dynamic-image generative system which can generate the dynamic image which changes 
in one and a method, and a storage according to advance of not only musical **** but musical 
sound. Another purpose of this invention is to offer a participation [ whose user the dynamic 
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image which was only excellent in a sense of togetherness with music can be danced based on 
musical-sound data again it not only carries out graphic display, but, and can set up the 
movement of a picture object like a hand freely ] type man machine interface. 
[0006] furthermore, in generating CG dynamic image based on musical-sound data There is a 
possibility of producing the delay of picture generation which cannot be disregarded since this 
picture generation is the reaction produced after generating of a musical-sound event, and in 
case it is interpolation Since animation speed may change with the load effects to CG drawing 
capacity and CPU of a computer or drawing of a key frame position may be flown CG animation 
may be unable to be created synchronizing with a musical performance, further In case CG 
dynamic image is used as a musical instrument player model, natural movement according to 
musical-sound data cannot be given to CG dynamic image only by controlling each part of a 
picture individually, only corresponding to each musical-sound data. Therefore, in view of many 
situations on such CG dynamic-image generation, still more nearly another purpose of this 
invention can avoid the delay of generation of a request picture, and can perform smooth 
interpolation processing according to the throughput of a system, and is to offer the new picture 
generation method which can also operate a player model with a natural performance gestalt by 
analyzing a settlement of musical-sound data further. 
[0007] 

[Means for Solving the Problem] In order to solve this technical problem, according to the 
musical-sound response picture generative system of this invention It corresponds to a 
performance data-processing means to output musical-sound control information and a 
synchronizing signal one by one corresponding to the musical piece which should be performed, 
and the aforementioned musical piece which should carry out a performance. A parameter supply 
means to supply the operational parameter for controlling the movement of each part of a 
picture, Based on a musical-sound generation means to generate musical sound based on the 
aforementioned musical-sound control information, the aforementioned musical-sound control 
information, and a synchronizing signal, it has a picture generation means to generate the picture 
by which movement is controlled according to the aforementioned operational parameter, by the , 
aforementioned picture generation means The picture which moves in accordance with advance 
of the musical-sound generation by the aforementioned musical-sound generation means is 
generated. 

[0008] Moreover, the step which outputs musical-sound control information and a synchronizing 
signal one by one corresponding to the musical piece which should be performed according to 
the musical-sound response picture generation method of this invention, The step which 
supplies an operational parameter corresponding to the musical piece which should be 
performed, the step which generates musical sound based on the aforementioned musical-sound 
control information, And based on the aforementioned musical-sound control information and a 
synchronizing signal, it consists of the step which generates the picture by which the movement 
of each part is controlled according to the aforementioned operational parameter, and a picture 
is moved according to advance of musical-sound generation. 

[0009] In the musical-sound response picture generation method by this invention Furthermore, 
before the information on a portion that musical sound and a picture should be generated among 
the aforementioned musical-sound control information in both the steps that generate the 
aforementioned musical sound and a picture is read, the information on the portion concerned is 
predicted and analyzed. The beat and the event should be suited in the movement of a picture 
by having the step which prepares the graphics data corresponding to the information on the 
portion concerned, and generating the aforementioned picture corresponding to the information 
on the portion concerned using this graphics data. 

[0010] According to another feature of this invention, according to the throughput of the system 
for generating the aforementioned picture, interpolation processing of the movement of each 
part of the aforementioned picture is carried out, and this interpolation processing uses the 
criteria key frame of the timing according to the synchronizing signal corresponding to musical 
piece advance, and is suitably realized by controlling the this key inter-frame number of times of 
interpolation according to system throughputs, such as machine speed. 
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[0011] According to still more nearly another feature of this invention, the aforementioned 
picture is used as a musical instrument player model, and after the performance gestalt which 
should take this musical instrument player model based on the aforementioned musical-sound 
control information is analyzed, the movement of each part of the aforementioned picture is 
controlled according to the analyzed performance gestalt. 

[0012] furthermore, to the storage for the musical-sound response picture generation by this 
invention While making musical-sound control information and a synchronizing signal output one 
by one corresponding to the musical piece which should be performed, supplying an operational 
parameter corresponding to the aforementioned musical piece which should carry out a 
performance and making musical sound generate based on the aforementioned musical-sound 
control information The picture by which the movement of each part is controlled according to 
the aforementioned operational parameter is made to generate based on the aforementioned 
musical-sound control information and a synchronizing signal, and the program for performing the 
process to which the picture was moved according to advance of musical-sound generation is 
memorized. 
[0013] 

[Function] Operation of the picture object displayed on a screen controls by acquiring the 
musical-sound control information and the synchronizing signal for controlling operation of each 
part of a picture object sequentially, and using CG technology using such information and a 
signal by interpreting the musical piece performance data showing the musical piece which 
should take out the conditions set up beforehand in this invention from the musical piece which 
should be performed, or should perform. 

[0014] MIDI (Musical Instrumentdigital Interface) performance data are used for musical piece 
performance data, and it is [ in / this invention ] effective for a picture object to consider as a 
3-dimensional (3D) picture using the dancer who dances synchronizing with such performance 
data. According to this invention, the picture which interprets the musical-sound control 
information included in MIDI performance data, and has movement autonomously can be 
generated, and changeful movement can be sequentially generated by applying a trigger to 
picture operation by an event, timing, etc. which were set up beforehand. 
[0015] In this invention, musical piece performance data like MIDI performance data are 
interpreted, it has the operational-parameter setting portion which determines operation and a 
sequence by setup of a user besides the engine portion which gives suitable operation (for 
example, dance) for a picture object, and the image which suits music exactly and carries out 
movement according to liking from these both can be generated. Therefore, it makes it possible 
to adopt a participated type and karaoke how to enjoy itself, and other MIDI performance data 
can enjoy a certain operational parameter. 

[0016] in this invention, not only enjoying the image which further only suited the music 
performance and it by MIDI performance data, but on a screen, rhythmical operation like a dance 
is danced and it performs to a hand (dancer) object — making — arbitrary setting change of an 
operational parameter — with [ this ] dance bending of hand — the pleasure which becomes a 
teacher can also be added and music business can be expanded by this 

[0017] In this invention, on the occasion of CG image processing of performance data, precede 
with advance of the musical-sound generation based on performance data, and performance data 
are predicted serially. While making " hard to be able to draw smoothly (picture generation), to 
be sufficient also for delay and " of drawing in a musical-sound generate time, and to produce by 
being made to perform CG analysis and prediction in advance corresponding to the event which 
a picture should answer A drawing processing burden is mitigated and much more natural 
operation can be made to perform a picture object 

[0018] In this invention, since it is made to carry out interpolation processing of the movement 
of each part of a picture by setting up the criteria key frame based on the predetermined 
synchronizing signal corresponding to musical piece advance, and using this criteria key frame on 
the occasion of CG image processing of performance data according to a picture generative- 
system throughput, operation of a smooth picture can be secured and, moreover, the animation 
which synchronizes with a musical performance can be created. 
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[0019] Since the performance gestalt which should take a musical instrument player model 
based on musical-sound control information is analyzed further and it is made to control the 
movement of each part of the aforementioned picture by this invention on the occasion of CG 
image processing of performance data according to the analyzed performance gestalt, a player 
model can create the animation which operates with reality with a natural performance gestalt 
[0020] 

[Embodiments of the Invention] Hereafter, the example of this invention is explained in full detail, 
referring to a drawing. In addition, in this invention, arbitrary concrete objects or abstract 
structure to give the movement doubled with music as a dynamic picture object can be adopted, 
for example, the person of a required number, an animal, vegetation, the structure, a pattern, etc. 
can use such combination arbitrarily. 

[0021] Reference of drawing 1 shows the hardware composition of the musical-sound response 
picture generative system by one example of this invention here. This system A personal 
computer (personal computer) system with a built-in sound source, Like the system which added 
the sound source and the display to the sequencer with a hard disk (personal computer) A 
central processing unit 0 [ CPU:central processing ] unit 1, a read only memory () [ ROM:read ] 
It has only memory2, RAM (RAM:random access memory) 3, an input unit 4, external storage 5, 
the input interface (I/F) 6, sound-source equipment 7, and display-processing equipment 8 
grade, these equipments It connects mutually through a bus 9. 

[0022] In drawing 1 , the predetermined program for controlling this system is memorized by 
ROM2, and the program about the various processings explained later is included in these 
programs. CPU1 performs various control according to the predetermined program memorized by 
ROM2 in this whole system, and carries out the sequencer and **** module function which are 
mentioned especially later pivotal. RAM3 is used as a work field for memorizing required data and 
a required parameter on the occasion of these control, and storing various registers, a flag, etc. 
temporarily. 

[0023] The input unit 4 possesses the control panel equipped with a keyboard, various switches, 
etc., and a coordinate position alter operation child like a mouse, and gives setting directions of 
various parameters of operation, a musical piece performance, and the directions about graphic 
display. For example, on a control panel, perform various number/symbol keys for inputting the 
operational-parameter set point, and a tempo rise / down (**5%), set the view (camera position) 
of 3-dimensional scenography all around, it is made to rotate, or required handlers, such as 
various function keys for making it return to a normal position, are prepared variously. Like 
keyboard form equipments, such as the further conventional electrohone and a synthesizer, this 
input unit 4 can be provided with musical piece performance data required for the graphic display 
which synchronized with this musical piece performance, while performing the musical piece 
performance by these keyboards etc. by having the keyboard and switch for a performance. 
[0024] External storage 5 is for memorizing or reading data, such as various data for CG, and 
background-image information, if needed out of the various parameters of operation which 
accompany musical piece performance data and this, and a floppy disk is used for a storage. 
[0025] The input interface 6 can be considered as the MIDI input interface which is an interface 
for receiving musical piece performance data from the external musical piece information source, 
for example, receives MIDI musical piece data from the external MIDI information source. In 
order to use the system of this invention itself as the information source to an external system 
of the same kind, an output interface is made to provide, and after changing musical-sound 
information or various accompanying data into predetermined data formats, such as for example, 
a MIDI format, the function sent out to an external system can be given to this input interface. 
[0026] Sound-source equipment 7 generates a digital musical-sound signal according to the 
musical-sound control information supplied through a bus 9, and supplies this to the musical- 
sound signal processor 10. After this signal processor 10 changes the supplied musical-sound 
signal into an analog musical-sound signal, it is made to pronounce by the loudspeaker 11. These 
musical-sound signal processors 10 and loudspeakers 11 constitute a sound system SP. 
[0027] Picture control information is supplied through a bus 9, and display-processing equipment 
8 generates a necessary video signal based on this picture control information, and carries out 
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graphic display of the picture corresponding to a display 12 with this video signal. These display- 
processing equipment 8 and displays 12 can constitute display system DP, and can give various 
image-processing functions, such as shading attachment to display-processing equipment 8. In 
addition, about the graphic display accompanying the expansion drawing processing and this to 
the picture of picture control information, the dynamic image which has a feeling of a lively 
motion and presence further can be indicated by visible by preparing independently the display- 
processing equipment or the large-sized display of exclusive use. 

[0028] The configuration of module of the musical-sound response picture generative system by 
one example of this invention is shown in drawing 2 , and it changes mainly from the sequencer 
module S, the sound-source module A, and the **** module I to it. 

[0029] The sequencer module S supplies musical-sound control information to the sound-source 
module A serially corresponding to the musical piece which should be performed. It is what 
supplies musical-sound control information and a synchronizing signal to the **** module L more 
specifically While outputting the musical-sound control information which chooses musical piece 
performance data like MIDI performance data, processes this, and corresponds From the clock 
signal used for selection and processing of musical piece performance data, the synchronizing 
signal corresponding to musical piece performance data is outputted, and it sends to the sound- 
source module A and the **** module I. 

[0030] The so-called "MIDI engine" which processes the musical piece performance data from 
the MIDI information source, and is given to a sound-source module can be used for this 
sequencer module S almost as it is. In addition, when using keyboard form equipments, such as 
an electrohone and a synthesizer, the data generation module which generates the above- 
mentioned musical-sound control information, information equivalent to a synchronizing signal, 
and a signal can be formed in these keyboard type equipments, and such a data generation 
module can be used for them as a sequencer module S. 

[0031] The sound-source module A is a module for generating a musical-sound signal based on 
the musical-sound control information received from the sequencer module S, and generating 
musical sound by the sound system SP t and can use the sound-source module in the 
conventional electrohone, automatic performance equipment, a synthesizer, etc. 
[0032] The **** module I is a module for creating picture control information based on the 
musical-sound control information and the synchronizing signal which are received from the 
sequencer module S, dancing on the display screen of display system DP, and displaying a 3- 
dimensional picture object like Hand D, and controlling the operation in picture generation mode. 
The **** module I is equipped with the parameter setting sub module PS again, and this sub 
module PS has the function to set up the operational parameter for controlling operation of each 
part of such a picture object D, in parameter setting mode. Therefore, the **** module I can 
control sequentially operation of each part of the picture object D by answering musical-sound 
control information and a synchronizing signal, and referring to a correspondence operational 
parameter, and it can synchronize with advance of the musical-sound generation by the sound- 
source module A at the picture object D, and arbitration adjustable operation according to a 
setup of an operational parameter can be made to be performed. 

[0033] The example of a picture displayed on a display screen in picture generation mode is 
shown to the **** outline target, and the main dancer MD and two persons 1 back dancers BD1 
and BD2 are used for drawing 3 as a 3-dimensional dynamic-image object in this example. I will 
explain still more concretely the example which makes these dancers MD, BD1, and BD2 dance 
according to advance of the music hereafter performed by the sound system using the musical- 
sound control information acquired from MIDI performance data. 

[0034] Although this **** module I is equipped with the dance module DM which performs 
processing required to carry out motion control of each moving part of the dance hand which is 
a picture object sequentially according to advance of the musical-sound generation by the 
sound-source module A in picture generation mode, in order to set up beforehand the 
operational parameter which determines the method of operation of each moving part of such a 
dance hand, it has the dancer setting module as a parameter setting sub module PS. This module 
supports a setup of a dancer s operational parameter in the parameter setting mode called 
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"dancer setting mode" below. As shown in .dra_wjng .4 which made the example the main dancer 
MD who is one of the dance hands of these, as for each dancers MD, BD1, and BD2, each part, 
such as the head, the upper half of the body, a wrist, and a hand, is defined as moving part 
outside the elbow (elbow) section EL, the arm (arm) section AR, and Foot LG. In addition, if the 
data processing capacity of a system allows, further, it can divide into portions, such as a 
shoulder, a drum, and the waist, and let these be moving part if needed. 

[0035] [Parameter setup procedure] The outline of the configuration procedure by the dancer 
setting module performed in the dancer setting mode of the **** module I is shown in drawing 
5 , and a selection setup of operation of the dancer with each dancers performance data set 
[ who were set up and was correlation-set up ] up of predetermined moving part is performed to 
it in this mode. 

[0036] In dancer setting mode, as shown in drawing 5 , for every dancer, a dancer and 
performance data are associated with block DS 1 1, the item of each moving part of a dancer of 
operation is chosen with block DS 2, and each parameter, such as a performance DCH and an 
attenuation value, is set up for every item of operation with block DS 3. In this example, among 
the selected moving part, about Arm AR, operation is set up with block DS 12, the motion 
control of a nodulus unit is set up in detail with block DS 4, similarly, about Foot LG, operation 
can be set up with block DS 13 and the motion control of a nodulus unit can be set up in detail 
with block DS 5. In addition, it is possible to design so that the same detailed setup may be 
performed about each of other moving part, such as the elbow section EL, the head, the upper 
half of the body, a wrist, and a hand. 

[0037] The "dancer setting" dialog screen corresponding to the vertical block DS 1 which 
includes the blocks DS11, DS12, and DS13 of drawing 5 in drawing 6 is shown, to drawing 7 The 
"channel setting" dialog screen corresponding to block DS 2 is shown, to drawing 8 The "data 
selection" dialog screen corresponding to block DS 3 is shown, in drawing 9 , the "operation 
setting of arm" dialog screen corresponding to block DS 4 is shown, and the "operation setting 
of leg" dialog screen corresponding to block DS 5 is shown in dr aw ing 10 at it. 
[0038] Now, if this system is fed into dancer setting mode using an input unit 4, first, the dialog 
screen of drawin g 6 is displayed on a display 12, and a setup as which it is indicated in the 
vertical block DS 1 of dr awin g 5 can be performed. 

[0039] The "dancer 1 " on the dialog screen of drawin g 6 , and corresponding to the main dancer 
MD of drawing 3 , the back dancer BD 1, and the back dancer BD 2, In each columns D1, D2, and 
D3 of "a dancer 2" and "a dancer 3" The "data selection" button DB for performing correlation 
with each dancer and performance data corresponding to block DS 11, respectively, The 
"operation setting of leg" button LB for setting up step operation of the leg doubled with the 
beat corresponding to the "operation setting of arm" button AB for setting up the bilateral- 
symmetry nature of operation of an arm corresponding to block DS 12 and the block DS 13 is 
displayed. Moreover, "rotation" check box TC for carrying out a display setup of whether 
rotation operation of the "display" check box DC and each dancer for giving a setting indication 
of whether each dancer is projected on a display screen individually is carried out individually is 
formed in each of each column. In addition, a check of "rotation" check box TC performs 
rotation processing for carrying out graphic display as each whole dancer is rotating at 
predetermined speed (each Otachi-dai is turning like) in dance mode. 

[0040] The number setting display IR of intro noduli is for setting up the number of noduli of the 
intro section only at the time of bend operation, and displaying this. In order to read an 
operational parameter from a file, the "setting reading" button RB is formed, since an operational 
parameter is saved at a file, the "setting preservation" button MB is formed in the screen lower 
part, and the "O.K." button and "cancellation" button are also prepared in it. 
[0041] [Performance data selection configuration procedure] In the "dancer setting" dialog 
screen of drawing 6 , if the "performance data selection" button DB of the "dancer 1" column 
D1 is clicked, the "channel setting" dialog screen shown in drawing 7 is displayed on a display 
12, and can choose **, such as a kind of MIDI performance data corresponding to operation of 
the main dancer MD, and a channel, a beat type, as the bottom of assistance of this screen. 
[0042] The initial-setting parameter obtained by operation of the "setting reading" button RB is 
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displayed on drawing 7 , the various items of each moving part, such as a dancer's elbow (elbow), 
an arm (arm), a leg, the head, the upper half of the body, a wrist, and a hand, of operation are 
enumerated by the item column MT of a "channel setting" dialog screen of operation, and the 
"set" button SB and various parameters of operation are displayed on it corresponding to these 
items of operation. Like drawing __7 , a setting indication of the various parameters of operation 
can be given at the "data-type" column DT, the "channel" column CH ( the "beat output" 
column BO, the "attenuation" column radiographic, the "scale" column SC, and the "cut-off" 
column CO. In addition, about an initial-setting parameter, if needed, according to the 
fundamental operation pattern of the dancer corresponding to desired ****, the default setting 
parameter of each moving part is defined beforehand, and such a default setting parameter can 
be displayed on the time of dancer setting mode starting etc. using a suitable read-out means. 
[0043] In the example of 1 display of the initial-setting parameter shown in drawing 7 As 
opposed to the item of 16 of operation of "the left elbow (bending)" in the item column MT of 
operation, "a right elbow (bending)", "a left arm (bending)", — , "the head (longitudinal 
direction)", and "the head (it leans)" Channel 1CH-16CH of MIDI performance data is set up as 
a channel number Cn which should answer, respectively, also about which item of operation The 
data type Vd of the MIDI performance data which should answer is set as "note-on" data, and 
the "scale" value Vs is set as "1.0000", the "cut-off" value Vc is set as "6" by "6", and 
"attenuation" value Va did not set up the "beat output" value Vb. 

[0044] In order to change such an initial-setting parameter and to obtain a desired operational 
parameter, each item of the item column MT of operation of operation is further directed by the 
click of the corresponding "set" button SB. For example, when the "set" button SB of an item 
"a left elbow (bending)" of operation is directed, the "data selection" dialog screen for setting 
up each parameter, such as the channel number Cn and the attenuation value Va, about an item 
"a left elbow (bending)" of operation is displayed like drawing 8 . The "data-type" setting area 
DA which becomes this dialog from the "note-on" setting section NS, "control" selection 
setting section CS, and the "beat type" selection setting section BS, and the "channel 
selection" setting area CA are formed, and the "beat output-value" setting display BR, the "of 
operation attenuation value" setting display RR, the "of operation scale" setting display SR, the 
"cut-off" setting display CR, etc. are formed in other area. 

[0045] In the "data selection" dialog screen of drawing 8 , in order to carry out a selection setup 
of the kind of performance data to the item "a left elbow (bending)" concerned of operation, the 
corresponding data type Vd is chosen by directing any one of each of the setting sections NS, 
CS, and BS of the "data-type" setting area DA. The "note-on" setting section NS and 
"control" selection setting section CS As a data type Vd which moving part should answer, it is 
for carrying out a selection setup of the event Iv out of MIDI performance data, "control" 
selection setting section CS So-called "control change" "[1] Modulation taken up from the 
function", The selection directions of any one "control" data can be carried out among "[5] 
Portament/Time", — , "[94] Effect 4Depth", and this can be set up as an event Iv. In addition, it 
can consider as "note-on / OFF" selection setting section which carries out the selection 
directions of "note-on" or the "note-off" also with the "note-on" setting section NS, and it can 
constitute so that the response to "note-off" may be attained. 

[0046] Moreover, the "beat type" selection setting section BS is for carrying out a selection 
setup of any one "beat type" data Bt as a data type Vd which moving part should answer among 
the kinds of beat which consists of "1 beat unit <a down>", "1 beat unit <a rise>", "2 beat units 
<a down>", — , "2 nodulus units." 

[0047] The "channel selection" setting area CA is area for carrying out a selection setup of the 
channel number Cn which makes operation cause arbitrarily out of the channels CH1-CH16 of 
16. The channel number Cn by which a selection setup was carried out here points to any of the 
setting sections NS and CS of the "data-type" setting area DA they are, and as a kind of 
performance data, when a selection setup of Event Iv, i.e., "note-on" data, or the "control" data 
is carried out, it becomes effective. 

[0048] The "beat output-value" setting display BR prepared in the right-hand side of the "beat 
type" selection setting section BS of the "data-type" setting area DA makes the velocity value 
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Vb of a beat output a "beat output value", and is a viewport for [ of "0"- "127" (7 bits) ] setting 
up in the range. The beat output value Vb by which a selection setup was carried out here 
becomes effective when a selection setup of the "beat type" data Bt of this selection setting 
section BS is carried out. 

[0049] The "of operation attenuation value" setting display RR prepared in the lower part of a 
"data selection" dialog screen It is a viewport for [ of "0"- "127" (7 bits) ] setting up in the 
range about the attenuation value (velocity attenuation value) Va of operation which determines 
the rate to which moving part is returned toward an initial valve position (an angle is included). A 
desired attenuation value of operation can be displayed and set up by pointing to this viewport 
using an input unit 4, and operating a numerical keypad. The "of operation scale" setting display 
SR moreover, the scale of each moving part of the main dancer MD who is a 3-dimensional 
picture object, and the back dancers BD1 and BD2 ( drawing 3 ) of operation It is a viewport for 
setting up a standard value with the scale-factor value Vs set to "1.0000". the "cut-off" setting 
display CR It is a viewport for setting up the lower limit Vc of a velocity value (performance 
information value) reacted to operation, and all can display and set up a desired value by the 
same operation as the setting display RR. 

[0050] In the example of a display of dr awin g 8 , each data item set up in^a Jdata type" and the 
"channel selection" setting area DA and CA The setting mark shown by "-" mark is given to the 
viewport of those left-hand side, about "a left elbow (bending)" of "a dancer 1" Carry out a 
selection setup of the "note-on" data as an event Iv, and it is shown that it is in the state 
where a selection setup of the channel number Cn was carried out at "CH1." Since a selection 
setup of the "beat type" data Bt is not carried out in the "beat type" selection setting section 
BS, "127" of the beat output value Vb of the setting display BR is invalid. Moreover, as for the 
attenuation value Va of operation, the state where the scale value Vs of "the left elbow 
(bending)" of the main dancer MD of operation was set as the standard value "1.0000", and the 
cut-off value Vc was set as "6" by zero ("0"), respectively is shown in each setting displays RR, 
SR, and CR. 

[0051] When a setting change of the operational parameter of "the left elbow (bending)" of "a, 
dancer 1" is made when the "O.K." button or "cancellation" button was clicked, a setup is 
ended and the "O.K." button is directed although it returned to the "channel setting" dialog 
screen of drawing 7 , and not making a setting change with "cancellation" button, it is still the 
original initial-setting parameter. Similarly, a change setup can be carried out about other items 
of the item column MT of operation of operation at the parameter value of a request of a desired 
operational parameter. 

[0052] If a setup or check of all operational parameters about "performance data selection" of 
"a dancer 1" is finished and the "O.K." button of the "channel setting" dialog screen of drawing 
7 or "cancellation" button is clicked, it will return to the "dancer setting" dialog screen of 
drawing 6 . About other "dancers 1" and "dancers 2", a setup or check of an operational 
parameter about "performance data selection" can be performed according to the same 
procedure. 

[0053] In addition, in drawing 6 , if a "set" button is clicked after a return setup will be carried 
out at an initial-setting parameter and the operational parameter corresponding to all items of 
operation will click "reset" button, if "reset" button is double-clicked, a return setup of the 
operational parameter corresponding to the item of operation corresponding to a "set" button 
will be carried out at an initial-setting parameter. Moreover, if a "set" button is clicked after the 
operational parameter corresponding to all items of operation will be zero or un-setting up and 
will click a "clearance" button, if a "clearance" button is double-clicked, the operational 
parameter corresponding to the item of operation corresponding to a "set" button will be zero or 
un-setting up. 

[0054] [Configuration procedure of an arm and a foot of operation] In the "dancer setting" dialog 
screen of drawing 6 For example, if the "operation setting of arm" button AB of the "dancer 1" 
column D1 is clicked The "operation setting of arm" dialog screen shown in drawing 9 is 
displayed on a display 12, and operation of the main dancers MD arm AR ( dr^wing„4 ) can be 
set per nodulus about bilateral-symmetry nature to the bottom of assistance of this screen. 
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[0055] In the "operation setting of arm" dialog screen of drawing J) , operation of a dancer s arm 
(arm) is divided into an item which is called "operation according to right and left", the "right- 
hand axial symmetry 1", — , "left-hand point symmetry 1" about bilateral-symmetry nature, and 
is enumerated by the "operation setting of arm" item column AT. The setting display area AA for 
setting up and displaying operation of the arm about these bilateral-symmetry nature on every 
eight vibrant tune unit "01" of a top line - "08" is established in the right-hand side of the item 
column AT, therefore operation with dance mode becomes the repeat of eight vibrant tunes. In 
addition, a setup of the elbow (elbow) section EL and a hand of operation related to Arm AR can 
be included, and when this is unnatural, you may make it set up separately symmetric-property 
operation of the elbow section EL and a hand, although symmetric-property operation of Arm AR 
may be set to this "setup of an arm of operation." 

[0056] Priority is given to the parameter set up corresponding to operation of the arm 
enumerated by the "operation setting of arm" item column AT over the parameter set up using 
drawing 7 and the dialog screen of 8. Therefore, if "operation according to right and left" is set 
up as a parameter, in dance mode, an arm on either side is separately operated corresponding to 
MIDI performance data, and on the other hand, "right-hand axial-symmetry 1" - "the left-hand 
point symmetry 1" will be set up in order to operate the arm AR on either side symmetrically, 
that is, — if "the right-hand axial symmetry 1" is set up — dance mode — moving part — a 
right arm — a left arm — receiving — an axial symmetry and the case where it was operated in 
follower and "the left-hand axial symmetry 1" is set up — a left arm — a right arm — receiving 
— an axial symmetry — and it is operated in follower and — the case of "the right-hand point 
symmetry 1" — a right arm — a left arm — receiving — a point symmetry — and it is made to 
operate in follower — having — moreover — "the left-hand point symmetry 1 " — a left arm — 
a right arm — receiving — a point symmetry — and it is operated in follower 
[0057] In the example shown in draw ing 9 , it indicates that it is in "operation according to right 
and left" the established state of an "operation setting of arm" parameter operates [ operation ] 
an arm on either side separately about all eight noduli with "-" mark in the setting display area 
AA. If this setup is ended or checked and "O.K." or "cancellation" button is clicked, it will return 
to the original "dancer setting" dialog screen ( drawing 6 ). Similarly, "a setup of an arm of 
operation" can be performed about other dancers BD1 and BD2. 

[0058] Next, in the "dancer setting" dialog screen of drawi ng 6 , if the "operation setting of leg" 
button LB of the "dancer 1" column D1 is clicked, the "operation setting of leg" dialog screen 
shown in drawing 10 is displayed on a display 12, and operation of the main dancers MD foot LG 
( drawing 4 ) can be set as the predetermined movement which follows to beats, such as step 
operation, under assistance of this screen. 

[0059] In the "operation setting of leg" dialog screen of drawing 10 , operation of a dancers leg 
is divided into the movement of a leg, such as "performance data linkage", a "right step", — , "a 
step", and is enumerated by the "operation setting of leg" item column LT. The setting display 
area LA for setting up and displaying the movement of these legs on every eight nodulus unit 
"01 " of a top line - "08" is established in the right-hand side of the item column LT like the 
case of "a setup of an arm of operation", therefore operation with dance mode also becomes 
the repeat of eight noduli, and it doubles with operation of an arm. 

[0060] Priority is given over this also to the parameter set up corresponding to the movement of 
the leg enumerated by the "operation setting of leg" item column LT when becoming the 
movement which disagrees with the parameter set up using drawing 7 and the dialog screen of 8 
too. If "performance data linkage" is set up as a parameter, although a leg will be interlocked 
with MIDI performance data in dance mode, "right step" - "a step" is used for setting up the 
movement of the decided leg according to a beat. 

[0061] About operation of a leg according to a beat, if a "right step" is set up Beginning to kick 
a right leg to the left to the right in half-a-step movement and "a right kick", when half-a-step 
movement and a "left step" are set to the right in dance mode, in "a left kick", it is begun to the 
left to kick a left leg, it moves to the right 1 step in "right translation", and moves to the left 1 
step by "left translation." Moreover, when the "before step right leg" was set up and half-a- 
step ****** return and a "before step left leg" are set from a right leg to a front, it moves to a 
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front from a left leg, and in the case of half-a-step ****** return and a "front move right leg", it 
moves to a front 1 step from a right leg, and returns, and in the case of a "front move left leg", 
it moves to a front 1 step and returns from a left leg. Furthermore, when the "after step right 
leg" was set up and half-a-step ****** return and a "after step left leg" are set to back from a 
right leg, it moves to back from a left leg, and in the case of half-a-step ****** return and a 
"back move right leg", it moves to back 1 step from a right leg, and returns, and in the case of a 
"back move left leg", it moves to back 1 step and returns from a left leg. And when a "bend" is 
set up, movement which bends both knees (knee) on that spot is carried out and "a step" is set 
up, it comes in dance mode to a standstill on that spot. 

[0062] In the example shown in d rawin g 1 0 , it indicates that the established state of an 
"operation setting of leg" parameter is in "performance data linkage" which interlocks a leg with 
MIDI performance data about all eight noduli with "-" mark in the setting display area LA. If this 
setup is ended or checked and "O.K." or "cancellation" button is clicked, it will return to the 
original "dancer setting" dialog screen ( drawing 6 ). Similarly, "a setup of a leg of operation" can 
be performed about other dancers BD1 and BD2. 

[0063] If various parameters are set up according to the musical piece which should be 
performed as mentioned above, by clicking the "setting preservation" button MB of the "dancer 
setting" dialog screen of drawing 6 , a musical piece name, ****, etc. can be attached and a 
series of set-up parameters can be saved at a file (external storage 5). Thus, the number of 
dancers can set up the main dancer MD, the back dancer BD 1, BD2 grade, and two or more 
person number, and can perform an individual setup to each part ( dr awin g 7 ) of each dancers 
body. Moreover, it is also possible to establish the setting means of **, such as a dress, a color 
the skin's, a hairstyle, and sex, and each dancers appearance parameter, and to generate a 
suitable picture to the music performed if needed. 

[0064] [Picture generation procedure] As shown in d rawing 1 1 , the dance module DM which 
performs processing which controls sequentially operation of the dancer who is a 3-dimensional 
picture object according to music can show the main function of the **** module I. In the dance 
mode of the picture module I, from the sequencer module S, this dance module DM receives , 
synchronizing signals, such as musical-sound control information like MIDI performance data and 
a beat (beat) timing signal, and a nodulus timing signal, and controls sequentially operation of 
each moving part of a dancer displayed on a display 12 synchronizing with advance of a 
performance of a musical-sound control signal according to the set-up parameter. 
[0065] The performance data-processing flow SM by the dance module DM is shown in drawing 
1.2 . This performance data-processing flow SM is performed in dance mode, and about a 
selection setup ( drawingJ7 "data-type" column DT) of a data type Vd, when the operational 
parameter corresponding to Event Iv, i.e., note-on, or control is set up ( drawing 8 setting 
sections NS and CS), it is applied. Therefore, this processing flow's SS reception of event 
information (MIDI performance data) starts it Then, I will explain the content of processing in 
each step of the processing flow SM first. 

[0066] [Step SM 1] The moving part of the dancer by whom the same channel number Cn as the 
received channel of MIDI performance data is set up is detected. 

[Step SM 2] It distinguishes whether about the moving part detected at a step SM 1 by the step 
SM 2, the set-up parameter is investigated and Event Iv is set up. here, Event Iv is set up — 
**** (YES) — it progresses to a step SM 3 and progresses to a step SM 10 at (NO) case [ to 
which Event Iv is not set ] 

[0067] [Step SM 3] At a step SM 3, the division of the present nodulus is carried out by 8, and 
the remainder is computed as a value which expresses the present nodulus unit (beatnow) Nm 
(Nm:0-7) about eight nodulus units (refer to drawing 9 "01" - "08"). 

[Step SM 4] At a step SM 4, it distinguishes whether in the present nodulus unit Nm which 
investigated the set-up operational parameter and was computed at the front step SM 3, the 
setup of symmetrical operation is carried out about the moving part concerned, here, 
symmetrical operation is set up — **** (YES) — it progresses to a step SM 5 and progresses 
to a step SM 10 at (NO) case [ to which symmetrical operation is set ] 

[0068] [Step SM 5] At a step SM 5, it checks whether the parameter set as the moving part 
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concerned is in agreement with the event Iv of receiving MIDI performance data further. Here, 
when it is checked that it is in agreement (YES), it progresses to a step SM 6, and it progresses 
to a step SM 10 at (NO) case. [ by which coincidence is not checked ] 

[0069] [Step SM 6] At a step SM 6, it distinguishes whether the velocity value (only henceforth 
a ''performance data value") Vm of the MIDI performance data received at a step SM 1 is 
beyond the setting cut-off value Vc. Here, in beyond (YES) the setting cut-off value Vc, it 
progresses to a step SM 7, and it progresses to (NO) case at a step SM 10. [ of under the 
setting cut-off value Vc ] 

[0070] [Step SM 7] At a step SM 7, about the moving part concerned, the amplitude value Am of 
operation is calculated from "performance data value" Vmx "of operation scale value" Vs= "of 
operation amplitude value" Am, and it moves to the purpose position Po displaced by this 
amplitude value Am of operation from the current position, and the moving part concerned is 
displayed on this purpose position Po, and is made finishing [ processing of the moving part 
concerned ]. 

[0071] In addition, it is not made to move to the purpose position (Pm) immediately as mentioned 
above, but you may make it move within predetermined timing by making this into a target 
position at the move display step which a musical piece is answered [ step ] and operates 
moving part positively like this step SM 7, interpolating toward this target position (Pm) from the 
current position. In this case, it is desirable to form a flag for every moving part during 
interpolation, until moving part reaches a target position (Pm), and to enable it to grasp the move 
state of moving part. 

[0072] [Step SM 8] At a step SM 8, the operational parameter related to the moving part 
concerned is investigated, and it distinguishes whether a setup of symmetrical operation is 
carried out to the moving part concerned and the symmetrical moving part in symmetric relation 
in the present nodulus unit Nm. Here, when symmetrical operation is set up (YES), it progresses 
to a step SM 9, and it progresses to a step SM 10 at (NO) case. [ to which symmetrical 
operation is not set ] 

[0073] [a step SM 9] — the amplitude value Am of operation calculates from "performance data 
value" Vmx "of operation scale value" Vs= "of operation amplitude-value" Am mentioned above, 
and it moves to the aforementioned moving part and a symmetrical target by this amplitude 
value Am of operation from the current position at purpose position Po 1 displaced (namely, — a 
part for -AM), and symmetrical moving part displays on this purpose position Po\ and makes 
finishing [ processing of the moving part concerned ] at a step SM 9 In addition, it can be made 
to move within predetermined timing by making purpose position Po 1 into a target position like a 
step SM 7 also here, interpolating toward this target-position Po' from the current position. 
[0074] [a step SM 10] — finishing [ processing ] in a step SM 10 — it is not — when it 
investigates whether there is still any moving part operated in the event Iv of receiving MIDI 
performance data about the remaining moving part and there is corresponding moving part (YES), 
one or less step [ SM ] processing is repeated about the moving part which returns and 
corresponds to a step SM 1 Moreover, it returns to the original state of standing by the next 
MIDI performance data reception to (NO) case without the moving part applicable to this. 
[0075] [Example of a performance data-processing flow at the time of individual operation] Next, 
an example of a processing flow to a predetermined operational parameter is explained. If an 
input unit 4 is operated, a MIDI file is loaded to a system based on the operating procedure 
which is not illustrated and a desired musical piece is chosen from this file, a series of 
operational parameters which correspond according to this musical piece will be read on RAM3. I 
will explain these operational parameters hereafter that it is a thing as displayed on drawin g 6 - 
10. In addition, since, as for "dancer 1" - "a dancer 3", the "display" check box DC of drawing 6 
is checked in this case, although it is selected by the object which should be processed as a 
picture object displayed, since "rotation" check box TC is not checked, rotation processing is 
not performed in graphic display. 

[0076] If the musical piece performance based on MIDI performance data is started based on the 
operating procedure which is not illustrated, the received MIDI performance data will be 
investigated and "a left elbow (bending)" of "a dancer 1" will be detected as moving part to 
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which same channel number Cn=CH1 as the channel CH1 of MIDI performance data is first set 
by the step SM 1. Since the "note-on" event Iv of MIDI performance data is set to this "a left 
elbow (bending)" as a data-type value Vd, it is judged as "YES" at the following step SM 2, and 
it progresses to a step SM 4, after computing the present nodulus unit Nm at a step SM 3. 
[0077] Since "operation according to right and left" ( drawing 9 ) is set to "a left elbow 
(bending)" about the arm related to an elbow (elbow) and symmetrical operation is not set up 
After coincidence with the "note-on" event Iv and the "note-on" event Iv of receiving MIDI 
performance data which were judged to be "YES" at a step SM 4, and were set as "the left 
elbow (bending)" at a step SM 5 is checked, it progresses to a step SM 6. 

[0078] It sets to a step SM 6 and is the velocity value (here) of receiving MIDI performance data. 
Since it is "note-on", in the usual case that the volume value Vm is larger than setting cut-off 
value Vc= "1.0000" It is judged as "YES" and the variation rate of the "left elbow" is carried out 
to the purpose position Po where only the angle which is equivalent to the size of 
VmxVs=Vmx1.0000=Vm from the current position (it is an initial valve position as shown to 
dra wing 13 (a) by this example) was bent in the following step SM 7. Therefore, as this "left 
elbow" is shown in drawing 13 (a), a left arm is bent in this purpose position Po, and a left arm is 
displayed. And it progresses to the following step SM 8. 

[0079] Since symmetrical operation is not set to "the left elbow (bending)" as mentioned above, 
after being judged as "NO" at a step SM 8, in a certain case, the moving part which should be 
operated in the event Iv of receiving MIDI performance data by the step SM 10 still returns to a 
step SM 1, the moving part which should next be processed is detected and the same 
processing is repeated about this moving part. 

[0080] [Example of a performance data-processing flow at the time of symmetrical operation] By 
"setup of an arm of operation" ( drawing 9 ) temporarily here for example When "a left arm 
(width)" is detected as moving part at a step SM 1 noting that the "left-hand axial symmetry" 
was set up as an operational parameter In a step SM 4, it is judged as "NO", returns to a step 
SM 1 through a step SM 10, and is excepted from individual processing of moving part. 
Therefore, at this time, the left arm of "a dancer 1" does not answer the "note-on" event [v.* 
[0081] However, when "a right arm (width)" is detected as moving part next at a step SM 1, in a 
step SM 4, it is judged as "YES", a step SM 5 and a step SM 6 are passed, and "a right arm 
(width)" is first moved horizontally by operating-value"Am" in a step SM 7. And "a left arm 
(width)" is moved to "a right arm (width)" and an axial symmetry by operating-value"-Am" in 
the step SM 9 after passing through a step SM 8 as symmetrical moving part which should follow 
on "a right arm (width)." Therefore, these "right arms" and "left arms" are displayed on purpose 
position Po symmetrical with an operating value "Am" and each who moved by "-Am", 
respectively, as shown in drawing 13 (b). And it progresses to the following step SM 10 which 
judges whether there is any moving part which should be processed further. 

[0082] Thus, after all processings of the moving part of "dancer 1" - "a dancer 3" which should 
answer the event of receiving MIDI performance data are completed Arrival of the following MIDI 
performance data by performing performance data processing of drawing 7 sequentially for every 
reception of waiting and MIDI performance data "Dancer 1 " - "a dancer 3" displayed on a 
display 12 can be danced according to advance of the musical piece performance by MIDI 
performance data. 

[0083] [Beat procedure] The beat processing flow SS by the dance module is shown in drawing 
14 . This beat processing flow SS is performed in dance mode, and about a selection setup of a 
data type Vd ( drawin g 7 DT), when the operational parameter of the "beat type" data Bt 
( drawing 8 "beat type" selection setting section BS) is set up, it is applied. Therefore, the 
moving part by which this processing is performed can perform rhythmical operation according to 
the beat suitable for advance of the musical piece performance by MIDI performance data. 
[0084] This beat processing SS synchronizes with the beat timing accompanying the musical 
piece performance by MIDI performance data, and is periodically started during the musical piece 
performance by MIDI performance data by the beat timing signal which moreover has the 
resolution of the double not less of this beat timing. By adopting such resolution, the upbeat and 
Down Beat (table [ of a beat ], timing in the reverse side) adjustment can be performed. 
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Processing at each step in this beat processing flow SS is as follows. 

[0085] [a step SS 1] — a ****** [ that it is the head of a nodulus at a step SS 1 when a beat 
timing signal is received ] — distinguishing — the head of a nodulus — it is (YES) — it 
progresses to a step SS 2 and (NO) case progresses to a step SS 3 [ which is not so ] 
[Step SS 2] At a step SS 2, after adding 1 to the several nm present nodulus and updating the 
number of noduli ("nm+1" ->nm), it progresses to a step SS 3. 

[Step SS 3] At a step SS 3, a beat type ( dr awing 7 BS) operational parameter is investigated, 
and the moving part set up so that it might answer to the timing of beat timing signal reception 
is detected. 

[0086] [Step SS 4] At a step SS 4, it distinguishes whether the present number Nt of beats of 
the detected moving part is "0", when it is "0" (YES), it progresses to a step SS 5, and it 
progresses to (NO) case at a step SS 8. [ which is not so ] 

[Step SS 5] At a step SS 5, the number Nt of beats of the moving part concerned is replaced by 
the set-up beat unit Nb ("Nb" ->Nt). 

[0087] As an operational parameter of the "beat type" data Bt (drawing 8 BS) in the set-up beat 
unit Nb here for example, when "1 beat unit (down)" is set up Value "Nb" =1 is taken, and also 
at the time of "beat unit (rise)" setup, it is "Nb" =1 and is set to "Nb" =3 also "Nb" =3 and "per 
2 beats (rise)" "per 2 beats (down)". [ 1 ] Similarly, when "3 Beat unit" is set up, value "Nb" =5 
are taken and it is "Nb" =7 at the time of "4 beat unit" setup. That is, since a rise and a down 
are contexts with the beat of performance timing, they are not influenced by Nb value. Moreover, 
about "1 Nodulus unit" and "2 nodulus units", it is set to "Nb" =nb~1 and "Nb" =2nb-1 
according to the number nb of beats for one nodulus, respectively. 

[0088] [Step SS 6] At a step SS 6, remainder which carried out the division of the present 
nodulus by 8 is made into the value showing the present nodulus unit Nm (beat now), and is 
computed. 

[Step SS 7] At a step SS 7, it distinguishes whether in the present nodulus unit Nm which 
investigated the operational parameter and was computed at the front step SS 6, the setup of 
symmetrical operation is carried out about the moving part concerned, here, symmetrical , 
operation is set up — **** (YES) — it progresses to a step SS 9 and progresses to a step SS 
13 at (NO) case [ to which symmetrical operation is set ] 

[0089] [Step SS 8] On the other hand, at a step SS 8, only 1 subtracts the number Nt of beats 
of the moving part concerned, and after updating ("Nt-1" ->Nt), it progresses to a value "Nt-1" 
at a step SS 1 3. 

[0090] [Step SS 9] At a step SS 9, it distinguishes whether the beat output value Vb is beyond 
the setting cut-off value Vc, when it is beyond the cut-off value Vc (YES), it progresses to a 
step SS 10, and in being under the value Vc, it progresses to a step SS 13. Since this step SS 9 
is a step for a check, it can be skipped if needed. 

[0091] [Step SS 10] At a step SS 10, about the moving part concerned, the amplitude value As 
of operation is calculated from "beat output-value" Vbx "of operation scale value" Vs= "of 
operation amplitude value" As, and it moves to the purpose position Po displaced by this 
amplitude value As of operation from the current position, and the moving part concerned is 
displayed on this purpose position Po, and is made finishing [ processing of the moving part 
concerned ]. This step is the same processing as the step SM 7 of performance data-processing 
SM, Therefore, it can be made to move within predetermined timing, interpolating toward this 
target position Po by making the purpose position Po into a target position similarly from the 
current position, and it is desirable for moving part to form a flag and to enable the grasp of the 
move state under interpolation in this case. 

[0092] [Step SS 1 1] At a step SS 1 1, like a step SM 8, the operational parameter related to the 
moving part concerned is investigated, and it distinguishes whether in the present nodulus unit 
Nm, a setup of symmetrical operation is carried out to the moving part concerned and the 
symmetrical moving part in symmetric relation. Here, when symmetrical operation is set up 
(YES), it progresses to a step SS 12, and it progresses to a step SS 12 at (NO) case. [ to which 
symmetrical operation is not set ] 

[0093] [Step SS 12] Also at a step SS 10, the amplitude value As of operation is calculated like 
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a step SM 9 from "beat output-value'' Vbx "of operation scale value'' Vs= "of operation 
amplitude value" As mentioned above, and it moves to purpose position Po' symmetrically 
displaced by this amplitude value As of operation from the current position, and symmetrical 
moving part is displayed on this purpose position Po\ and is made finishing [ processing of this 
symmetrical moving part ]. In addition, it can be made to move within predetermined timing, 
interpolating toward this target-position Po' by making purpose position Po' into a target position 
at this step as well as a step SS 10 from the current position. 

[0094] [a step SS 13] — finishing [ processing ] in a step SS 13 — it is not — when it 
investigates whether there is still any moving part operated to the timing concerned about the 
remaining moving part and there is corresponding moving part (YES), two or less step [ SS ] 
processing is repeated about the moving part which returns and corresponds to a step SS 3 
Moreover, it returns to the original state of standing by reception of the following beat timing 
signal to (NO) case without the moving part applicable to this. 

[0095] Since the beat processing flow SS consists of such a step SS 1 - a step SS 13 By this 
beat processing SS, as an operational parameter of the "beat type" data Bt (drawing 8 BS), 
when "1 beat unit (down)" is set up That the variation rate of the moving part is carried out can 
understand easily only the amount according to the beat output value Vb and the scale value Vs 
of operation which were set up for every down timing of one beat. 

[0096] [Attenuation procedure] The attenuation processing flow SA by the dance module is 
shown in drawing 15 as "attenuation processing (I)." The attenuation processing flow SA is for 
performing attenuation operation to which the moving part by which answered the performance 
on-going musical piece by each processing of drawing 12 and drawing 14 , and the variation rate 
was carried out from the initial valve position (an angle is included) is gradually moved so that it 
may return toward an initial valve position from the current position, therefore may be good. [ of 
return processing ] 

[0097] The attenuation processing SA can be started by periodical interruption during the 
musical piece performance by MIDI performance data. Starting of the attenuation processing SA 
is performed by the attenuation timing signal of the comparatively long repeat period of the » 
grade which does not become unnatural visually, it is necessary to synchronize this timing signal 
with beat timing, and it does not need to synchronize becoming independent with beat timing. 
Processing at each step in the above-mentioned attenuation (processing I) flow SA is as follows. 

[0098] [Step SA 1] If an attenuation timing signal is received, the current position of each 
moving part will be investigated at a step SA 1, and the moving part from which the position has 
shifted [ initial valve position ] will be detected. Although the natural-posture posture into which 
it considered as the position of the most natural and stable moving part which suited the dance 
of a musical piece, for example, the dancer stood straight like drawin g 4 in this example explains 
the initial valve position used as the criteria position of this detection, let them be other 
arbitrary positions if needed. 

[0099] [Step SA 2] At a step SA 2, the distance L between the current position and an initial 
valve position is computed as a position error of the detected moving part. 

[Step SA 3] At a step SA 3, it asks for unit travel Lu=La/(alphaVa) [the transformation constant 
as which alpha was determined suitably] using the attenuation value Va of operation acquired 
from the operational parameter set as the moving part concerned, and move the moving part 
concerned to the position which only this unit travel Lu displaced toward the initial valve position 
from the current position, it is made to display on this position, and attenuation operation of the 
moving part concerned is made finishing [ processing ]. 

[0100] [a step SA 4] — finishing [ processing ] in a step SA 4 — it is not — when it investigates 
whether there is still any moving part which carries out attenuation operation at the time 
concerned about the remaining moving part and there is corresponding moving part (YES), one or 
less step [ SA ] processing is repeated about the moving part which returns and corresponds to 
a step SA 1 Moreover, it returns to the original state of standing by reception of the following 
interrupt signal to (NO) case without the moving part applicable to this. 

[0101] In order to explain this attenuation processing SA briefly, operation of the dancer by 
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whom attenuation processing SA is performed is drawn on drawing 13 (c) by the **** outline 
target. For example, supposing an initial valve position is a position shown with an alternate long 
and short dash line t a dancer's "a left arm (width)" When it is in the current position which it is 
at the attenuation timing signal reception time, and is shown with the dashed line of drawing 13 
(c), "A left arm (width)" is detected as moving part at a step SA 1, the distance L between the 
current position of "a left arm (width)" and an initial valve position is computed at a step SA 2, 
and it sets to a step SA 3. The attenuation value Va of operation of the operational parameters 
of "a left arm (width)" is investigated (value of the drawing 7 "attenuation" column radiographic 

6 '). Value L/Va=L / 6 which carried out the division of the distance L by these of operation 
attenuation value Va=6 are calculated, and "a left arm (width)" is moved to the position of the 
solid line which displaced only unit travel Lu=La/6alpha in the direction of the initial valve 
position of an alternate long and short dash line. 

[0102] As already stated, at performance data-processing SM and a move display step like 
Steps SM7, SM9, SS10, and SS12 in the beat processing SS, processing operation of 
interpolation movement is employable, and according to Event Iv or Beat Bt, moving part can 
indicate by move more by this at nature rather than is momentary. For example, another routine 
put side by side to these move display steps performs, it is made to move, interpolating moving 
part toward this target position from the current position, and processing of this interpolation 
movement is ended when moving part reaches a target position. Moreover, in connection with 
this, during interpolation, a flag is formed for every moving part until moving part reaches a 
target position (Pm), and it enables it to grasp the move state of moving part. 
[0103] A place which the attenuation processing flow SA which another attenuation processing 
flow SA by the dance module is shown as "attenuation processing (II)", and was shown here is 
applied to drawing 16 when adopting the above interpolation move processings, and is different 
with "attenuation processing (I)" of drawing 15 is that "a step SA 1-2" is inserted among steps 
SA [ SA1 and ] 2 with adoption of interpolation. 

[0104] [Step SA 1-2] At this step SA 1-2, it is judged about the moving part by which it was 
detected that the current position has shifted from the initial valve position at a step SA 1 
whether this moving part is interpolation moving. Here, if it is under movement, other moving part 
which should be progressed and decreased to a step SA 4 will be investigated. Moreover, if it is 
not [ be / it ] under movement, it will progress to a step SA 2 and attenuation processing will be 
performed. In addition, in order to grasp the move state for example, under interpolation, the flag 
formed for every moving part can be used for judgment whether moving part is interpolation 
moving. 

[0105] Thus, operation of each moving part of a dancer is sequentially controllable by three 
processings SM, SS, and SA synchronizing with advance of a performance of a musical-sound 
control signal. Moreover, since consecutive processing of the calculation value is diverted and 
carried out like Steps SM7-SM9 and Steps SS10-SS12 about symmetrical operation of the 
moving part which can direct colorful operation and is in symmetric relation since a certain 
moving part is made to answer an event (Iv) by performance data-processing SM and each 
moving part is made to answer a beat (Bt) by another beat processing SS, processing 
composition is simplified. 

[0106] Furthermore, about return operation, natural operation called an original-state return is 
realizable by using the attenuation processing SA of easy processing composition to positive 
operation which answers the event and beat of a musical piece by Processings SM and SS. A 
stable and natural position can be made to carry out the variation rate of each part of a dancer 
for a criteria position as an initial valve position like dra win g 4 further again about the criteria 
position (for an angle to be included) of the variation rate of these positive operation, i.e., the 
variation rate in Steps SM7, SM9, SS10, and SS12. 

[0107] As mentioned above, although various conditions were specified about the parameter 
setting mode (dancer setting mode) and picture generation mode (dance mode) in the case of 
using a dancer as a picture object and the example accompanied by easy CG operation has been 
explained, it cannot pass over this example to an example to the last, but it can perform change 
and addition if needed within the limits of the thought of this invention. 
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[0108] For example, although the criteria position was made into the initial valve position in the 
example for the simplification of operation about the criteria position in front of displacement of 
the picture object moving part in Steps SM7, SM9, SS10, and SS12 (an angle is included) Since 
movement of a picture object is made [ more complicated ] colorful, the newest position which 
displaced this criteria position as the current position with the big velocity value beyond a 
predetermined value in enlarging change of movement can be made into the movement which 
MERIHARI attached to movement as a criteria position. 

[0109] In addition, about the graphic display to a display screen, various picture setup, an image 
processing, and image ornamentation can be adopted outside rotation ( drayy|ng.6 TC <rotation 
processing» of the picture object mentioned already, and the image effect can be made colorful. 
For example, the reflection and shading attachment by the lighting and lighting which have 
change from the movable light source of 1 or a large number change of the camera (view) 
position which enabled it to set up appearance, such as a dress, a color of the skin, a hairstyle, 
and sex, about the picture object itself, and was mentioned already about the image processing, 
and outside rotation ( drawing 6 TC <rotation processing» of a picture object can be performed. 
Furthermore, as colors, such as the light source and a background image, lightness, a camera 
position (zoom), etc. are changed according to musical-sound control information or a 
synchronizing signal, or the suitable function key of an input unit 4 ( drawing ! ) is operated on 
the occasion of graphic display and various image operations are performed artificially, the much 
more colorful image effect can be acquired. 

[0110] moreover, about the concrete application to CG image processing of performance data 
For example, by preceding with advance of the musical-sound generation based on performance 
data a little, predicting performance data serially, and performing CG analyses (size of the 
amount of data etc.), and prediction in advance The lap of the processing which it is sufficient 
also for " and is called " is prevented, or certainty the synchronization with the musical sound 
generated and synchronous [ between each moving part ] can be raised further. 
[0111] Such read-ahead technology can be applied or transformed, a picture object can be 
controlled in prediction using another analysis result from performance data, and an advanced, 
picture can be generated. For example, not only one event but two or more events are divided 
on a certain time-axis, and it is possible to predict the position of the moving part which plays 
the musical instrument from a set of the note number of "note-on" information. If an example is 
given, a chord is analyzed from a performance data distribution (it is (like "DO", "Ml", and 
"SO")), and if it is the scene on which picture objects, such as a dancer or a pianist, are playing 
the piano based on this, while expecting the position of the wrist, the remaining arm information 
will also be created. 

[01 12] moreover, even the purpose or target position (Po) called for at a move display step like a 
step SM 7 about the interpolation processing mentioned above Deduce the number of times of 
drawing from tempo information and animation speed, and carry out interpolation movement, or 
Interpolate even this target position (Po) by beat synchronization, it is made to make a picture 
object reach a target position one by one within predetermined timing, and, thereby, a precision 
of operation can be improved further. 

[0113] Furthermore, the musical instrument performance information in performance data, i.e., 
the so-called ^'program change" information in MIDI performance data, is received, and you may 
make it make it picture objects, such as a dancer, play a musical instrument using this 
performance information. For example, since there are a piano tone, a violin tone, etc. by the 
difference in this "program change" information even if it is the same "note-on" event, 
performance operation peculiar to a musical instrument corresponding to this information can be 
made to perform, about performance operation peculiar to such a musical instrument, the dancer 
setting module of an example is shown in drawing 10 — as — **** — since it has the template 
of a breast of operation, it is also possible to develop this template of operation and to specify a 
musical instrument here 

[0114] As read-ahead-analyzed, ] [performance data mentioned above, are in charge of the 
concrete application to CG image processing of performance data. Preceding with advance of 
the musical-sound generation based on performance data a little, predicting performance data 
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serially, and performing CG analysis and prediction in advance It is very advantageous to raising 
further certainty the synchronization with the musical sound which prevents the lap of the 
processing which it is sufficient also for " and is called ", or is generated, and synchronous 
[ between each moving part ]. In order to perform such read-ahead analysis, according to the 
desirable embodiment of this invention, the read-ahead pointer is prepared separately from the 
reproduction pointer of performance data, this read-ahead pointer is used by the application 
side, it precedes with the musical piece performance by the performance data concerned, and 
performance data are analyzed in advance. 

[0115] According to the suitable embodiment of this invention, read-ahead analysis of 
performance data is performed in d rawing 17 in dance mode, and the conceptual diagram showing 
theoretically the case where CG picture based on the analysis result is made to generate 
according to a musical piece performance is shown. As shown in this drawing, in the read-ahead 
analysis processing by this invention, two, the reproduction pointer RP and the read-ahead 
pointer PP, are prepared as a read-out pointer of performance data. The reproduction pointer 
RP is a pointer for position management of the data block under present reproduction of the 
performance data which consist of the performance data block DO, D1, D2, and — . The read- 
ahead pointer PP formed separately from this reproduction pointer RP It is a pointer for pointing 
to the data block which precedes only a predetermined number (n-m) as opposed to the 
reproduction data block directed by the reproduction pointer RP, and preparing CG data to the 
reproduction data block concerned. 

[01 16] If the musical piece which should be performed is chosen, before pronunciation directions 
of performance data come, the read-ahead pointer PP will predict the performance data 
concerned, will store in storage the result which started and analyzed the analysis, and will go. 
for example, a time — tm+1 the read-ahead pointer PP — the inside of performance data — 
data block Dm if directed — this data block Dm Performance data are analyzed. And it is tn+1 at 
the reproduction time of the performance data concerned, finding the required event 
corresponding to the specified operational parameter from this performance data, and using this 
event and its time as judgment material. CG data corresponding to the picture which should be 
set and generated are determined, and this is made to memorize as an analysis result And 
display system DP is made to draw CG picture which reads an analysis result from storage at 
the time of musical-sound generating of this performance data (tn+1), and corresponds to it. 
[01 17] One example of such a read-ahead analysis processing flow SE is shown in drawing 18 , 
and it changes from processing (A) with the read-ahead pointer PP, and processing (B) with the 
reproduction pointer RP to it. Although it is desirable for it to be necessary to start by periodical 
interruption, and to start processing (A) with the read-ahead pointer PP by periodical 
interruption, processing (B) with the reproduction pointer RP may be made to be started, when 
the burden of other important processings (for example, processing with a reproduction pointer) 
is heavy, it is not started but there is remaining power. 

[01 18] [Read-ahead pointer processing (A)] In the read-ahead analysis processing flow SE, first, 
by read-ahead pointer processing (A) which consists of each following steps SE1 1-SE14, 
drawing preparation is made in advance and reproduction pointer processing (B) is performed 
after that. 

[01 19] [Step SE 1 1] If read-ahead pointer processing (A) is started by receiving event 
information, the performance data of the data block portion directed by the read-ahead pointer 
PP at a step SE 1 1 will be detected, for example, drawing 17 — setting — a time — tm+1 Data 
block Dm directed by the read-ahead pointer PP Performance data are detected and it 
progresses to a step SE 1 2. 

[0120] [Step SE 12] Performance data Dm detected at a step SE 12 It is tn+1 at the 
reproduction time of the performance data concerned, finding the required event corresponding 
to the specified operational parameter, and using [ are analyzed, for example, ] this event and its 
time as judgment material from performance data. CG data corresponding to the picture which 
should be set and generated are determined. In addition, in the analysis in this step SE 12, it is 
the performance data Dm concerned. The result otherwise analyzed about performance data 
Dm-1, Dm-2, and — at the time of the read-ahead pointer processing performed before it can 
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be used, 

[0121] [Step SE 13] At a step SE 13, CG data determined as an analysis result in a step SE 12 
are stored in storage with the pointer concerned, and it progresses to a step SE 14. 
[Step SE 14] At a step SE 14, one read-ahead pointer PP is carried forward and it returns to 
the state of standing by the next interruption. 

[0122] [Reproduction pointer processing (B)] The reproduction pointer processing (B) performed 
after the read-ahead pointer processing (A) which consists of such steps SE1 1-SE14 consists 
of the following steps SE21-SE25. 

[0123] [Step SE 21] If reproduction pointer processing (B) is started by receiving event 
information in read-ahead pointer processing (A) with some delay, the performance data of the 
data block portion directed by the reproduction pointer RP at a step SE 21 will be detected, for 
example, drawing 17 — setting — a time — tn+1 Data block Dm directed by the reproduction 
pointer RP Performance data are detected and it progresses to a step SE 22. 
[Step SE 22] At a step SE 22, pronunciation processing and sound-source processing required 
in addition to this are immediately performed based on the detected performance data (for 
example, Dm). 

[0124] [Step SE 23] At a step SE 23, in parallel to the pronunciation processing in a step SE 22, 
the analysis result (CG data) prepared for [the step SE 12 of read-ahead pointer processing (A)] 
to the performance data concerned at the time of a read ahead is read from storage based on a 
reproduction pointer, and it progresses to a step SE 24. 

[Step SE 24] At a step SE 24, CG picture is drawn based on the read analysis result (CG data), 
and it progresses to a step SE 25. Consequently, on the screen of a display 12, the picture 
corresponding to the performance data (for example, Dm) concerned is displayed synchronizing 
with the performance. 

[Step SE 25] At a step SE 25, one reproduction pointer RP is carried forward and it returns to 
the state of standing by the next interruption. By such processing flow, the pronunciation and 
picture generation processing corresponding to performance data are performed one by one. 
[0125] Although intermediary explanation of a read ahead and the reproduction was given in the 
example mentioned above what carries out parallel processing on real time, you may be made to 
regenerate, after following all one musical pieces in a read ahead and preparing drawing 
preparation about all performance data before reproduction by carrying out batch processing of 
the performance data from a MIDI file. 

[0126] As mentioned above, " can be made hard to be able to perform smoothly the 
pronunciation and drawing (picture generation) in the reproduction time (at the time of event 
generating) based on the event concerned, to be sufficient also for delay and " of drawing, and 
to produce, since CG data corresponding to the event which a picture should answer are 
prepared beforehand according to read-ahead analysis processing of this invention. Moreover, 
since the drawing processing burden in a reproduction time is mitigated, when displaying a pianist 
as a picture object, in case [ in which I will draw / / the moving part at the time of event 
generating only as a pianist's right hand so that this right hand may be moved corresponding to 
the event concerned ] it carries out, the suitable distance of making the left hand which is not 
directly related to the event concerned raised can also be built with remaining power, for 
example. 

[0127] [Interpolation processing] If processing operation of interpolation movement is adopted in 
a move display step like Steps SM7, SM9, SS10, SS12, and SA3 in performance data-processing 
SM, the beat processing SS, and the attenuation processing SA of operation as mentioned 
above, it is very advantageous for indicating moving part by move automatically finely toward the 
purpose position according to performance information, such as Event Iv and Beat Bt, from the 
current position. According to another desirable embodiment of this invention, interpolation 
processing can be performed to these move display steps using the key frame set up 
corresponding to the predetermined synchronizing signal accompanying advance of musical piece 
performances, such as ** (beat) and a vibrant tune, and, moreover, interpolation control 
corresponding to the throughput of a picture generative system can be realized now. 
[0128] That is, according to interpolation processing of this invention, according to the 
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throughput of a picture generative system, it is suitably controllable in interpolation movement of 
moving part by installing separately the interpolation manipulation routine called "number-of- 
times control of the interpolation in the appointed time length" or "interpolation control by time 
collating" which performs interpolation processing using the above-mentioned key frame to 
these move display steps. In addition, on the occasion of interpolation processing of this 
invention, during interpolation, a interpolation flag is stood for every moving part until moving 
part arrives at the purpose position, by this flag, the moving part concerned is in a move state, 
and it is grasped that it is the object of interpolation processing. 

[0129] [Number-of-times control = interpolation processing (of interpolationl) in the appointed 
time length] First "number-of-times control of interpolation in the appointed time length" For 
example, specify time length beforehand by time bases, such as ** (beat) and a vibrant tune, and 
the criteria CG drawing timing in the timing corresponding to the specified time length is set up 
as the key frame kfi, kfi+1, and — (i= 0, 1 and 2, — ). The number of times of interpolation of 
time Osanai of each key frame is controlled. 

[0130] The timing diagram for explaining the case where carry out time specification length to 
drawing 1 9 per ** (beat), and such number-of-times control of interpolation is performed to it is 
shown. That is, in the example of the number-of-times control of interpolation shown in this 
drawing, the drawing key frame kfi, kfi+1, and — are updated corresponding to the performance 
timing bi specified by ******, bi+1, and — . Although interpolation operation is performed n times 
on the interpolation point cj G™ 1» 2, — , n) between these key frames If the feature of the 
number-of-times control of interpolation of this invention is followed, the number of times n of 
interpolation in the appointed time length (kfi— kfi +2, kfi+1 -kfi+2, — ) is suitably controlled 
according to the throughput of a system, and can perform interpolation with corresponding 
fineness. 

[0131] The example of an important section processing flow in such number-of-times control of 
interpolation is shown in drawing 20 as "interpolation processing (1)." The flow SN of this 
interpolation processing (1) is a thing about the example ** (beat) is specified to be as time 
length like drawing 19 , and is as follows. [ of processing at each step ] 

[0132] [Step SN1] This interpolation processing (1) is started by periodical interruption of the 
predetermined-time interval set up according to the throughput of a system, if CG picture object 
moving part to which the interpolation flag which shows that it is under interpolation is stood is 
detected, data required for interpolation control of performance information, control information, 
etc. about the moving part first detected in Step SN1 will be acquired, then it progresses to Step 
SN2. 

[Step SN2] — a ****** [ that the ** data in performance information are judged at Step SN2, 
and it is ****** timing ] — distinguishing — ****** timing — it is (YES) — it progresses to 
Step SN8 and (NO) case progresses to Step SN3 [ which is not so ] 

[Step SN3] At Step SN3, as compared with the number of times n of interpolation assumed by 
any value in the interpolation point number cj at the beginning, if it is cj>=n, it progresses to Step 
SN7, and when that is not right (cj<n), it will progress to Step SN4. 

[0133] [Step SN4] At Step SN4, about the moving part concerned, only 1 increments the 
interpolation point number cj, and after updating ("cj+1" ->cj), it progresses to a value "cj+1" at 
Step SN5. 

[Step SN5] at Step SN5 If key inter-frame movement magnitude from the original position of the 
key frame kfi concerned obtained by carrying out the multiplication of the predetermined 
coefficient Kn to the multiplication value (the amount of total displacement : an angle of rotation 
and a travel) of the velocity value V of performance information and the scale value Vs of 
operation further to the last position is made into An=KnxVxVs About the moving part 
concerned, by Anx(cj/n) = "interpolation variation" Vj, the interpolation variation Vj to a 
interpolation position is calculated from the original position of the key frame kfi concerned this 
time (the j-th time), and it progresses to Step SN6. 

[Step SN6] It returns to the state will stand by the next starting if it will return to Step SN1 if 
there is moving part of the degree to which a return is carried out after drawing in the 
interpolation position this time which displaced only the interpolation variation Vj from a position 
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at Step SN6 at the beginning and moving the moving part concerned to this position from a 
interpolation position last time (the j-1sttime), and the interpolation flag is stood, and processing 
same about the following moving part is performed and there is nothing In addition, A/n= "unit 
interpolation variation" Vu is calculated, and you may make it draw the position which carried 
out the variation rate only of the unit interpolation variation Vu from the interpolation position 
last time (the j-1st time) at Step SN6 at Step SN5. 

[0134] [Step SN7] At Step SN7, about the moving part concerned, only 1 increments the 
number-of-times variation r of interpolation, and after updating ("r+1"->r) t it progresses to a 
value "r+1" at Step SN5. 

[0135] [Step SN8] At Step SN8, the key frame kfi is updated ("kfi+1" ->kfi) and it progresses to 
Step SN9. 

[Step SN9] — a ****** [ that the number-of-times variation r of interpolation is "0" at Step 
SN9 ] — distinguishing — r= 0 — it is (YES) — it progresses to Step SN10, and when that is 
not right (NO:r> 0), it progresses to Step SN1 1 

[Step SN10] At Step SN10, the number of times n of interpolation is updated for the 
interpolation point number cj ("cj"->n), and it progresses to Step SN12. 

[Step SN1 1] At Step SN1 1, after updating the number of times n of interpolation to value n+r 
added to the number-of-times variation r of interpolation ("n+r"— >n) t it progresses to Step 
SN12. 

[Step SN12] At Step SN12, it is the following key frame kfi+1. In order to prepare for processing, 
after initializing the interpolation point number cj and the number-of-times variation r of 
interpolation to a value "0", respectively, it progresses to Steps SN4-SN6 about the moving part 
concerned. 

[0136] In addition, the number-of-times control of interpolation by interpolation processing (1) 
Since it is in updating the number of times n of interpolation at Steps SN10 and SN1 1 via the 
renewal step SN8 of a key frame so that it may explain in detail later In order to operate 
effectively this number-of-times control of interpolation irrespective of change of an interrupt 
time interval, it is necessary to make the interpolation section (total displacement time) from the 
current position of moving part to the purpose position straddle two or more key frames. 
Therefore, as for the coefficient Kn in Step SN5, it is desirable to make it less than one value. 
However, about predetermined moving part, a coefficient Kn can be made or more into one 
(under 1 key frame period) by composition which diverts the number of times n of interpolation 
updated about a certain moving part also to interpolation processing of other moving part in 
future key frames. 

[0137] According to this interpolation processing (1), so that I may be understood from the 
above steps SN1-SN12 : [1] to which the next operation is performed — the drawing key frame 
period kfi— kfi +1 concerned since it is updated by the correspondence drawing key frame kfi to 
the renewal timing bi of interpolation operation ****** — the following ****** timing Bi+1 Until 
it reaches (a) Until the interpolation point number cj of interpolation, i.e., the number of times, 
reaches the number of times n of setting interpolation If only this number of times cj of 
interpolation interpolates and the number of times cj of (b) interpolation exceeds the number of 
times n of setting interpolation by Steps SN2-SN6 Only the number of times r continues 
interpolation too much further by Steps SN5 and SN6, carrying out the increment ("r+1"->r) of 
the number-of-times variation r of interpolation one by one through Step SN7. 
[0138] [2] the following drawing key frame period kfi+1 -kfi+2 receiving setting operation — the 
following ****** timing Bi+1 If it reaches, the key frame kfi It is the following drawing key frame 
kfi+1 at Step SN8. It is updated, about the number of times n of interpolation (a) It is updating 
timing Bi+1 at the number of times cj of actual interpolation below the number of times n of 
setting interpolation. When it reaches (r= 0) It is updating timing Bi+1 at number-of-times n+r of 
actual interpolation which makes this number of times cj of actual interpolation the number of 
times n of setting interpolation, and exceeds the number of times n of (b) setting interpolation by 
Step SN10. When it reaches (r> 0), by Step SN11 It is the following drawing key frame kfi+2 
further, using this number-of-times n+r of actual interpolation as the number of times n of 
setting interpolation. Frame period kfi+1 -kfi+2 to updating While preparing for interpolation 
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operation About the interpolation point number cj and the number-of-times variation r of 
interpolation, it is initialized by "0" in Step SN12, respectively. 

[0139] That is, (a) frame period kfi-kfi +1 When interpolation operation which set and was 
actually performed is below the number of times n (r= 0) set up beforehand It is degree frame 
period kfi+1 -kfi+2 one by one in the interpolation point number cj cj, i.e., the number of times 
which was actually able to be interpolated, which reached with this frame noting that there was 
no remaining power of interpolation. As the number of times of setting interpolation, the number 
of times of interpolation is completed as the value according to the throughput of a system, 
(b) Frame period kfi-kfi +1 It considers that there was remaining power which only a 
predetermined number n interpolates and is interpolated r times also after that when it set and 
interpolation operation exceeded a predetermined number n (r> 0), and is the following frame 
kfi+2 further. It considers as the number of times of setting interpolation including this remaining 
power part r, and enables it to perform still finer interpolation to updating. Also in this case, the 
number of times of interpolation is converged on the value according to the throughput of a 
system, and fine interpolation processing is performed by this number of times of interpolation. 
[0140] Therefore, it becomes possible to interpolate the fineness according to the throughput of 
a system, and even if it is the same system, the correspondence which fluctuates the number of 
times of interpolation one by one from the drawing key frame which follows is realizable on real 
time according to interpolation processing (1) of this invention, to the change in the load which 
should be processed. Moreover, especially this interpolation processing (1) is suitable to acquire 
CG animation picture which synchronizes with a beat. 

[0141] [Interpolation control = interpolation processing (2) by time collating] Next, "interpolation 
control by time collating" While setting up the criteria timing corresponding to the time length D 
beforehand specified arbitrarily by time bases, such as ** (beat), a nodulus, and the number of 
tics, in the musical piece which should be performed as the key frame kfi, kfi+1, and — (i= 0, 1 
and 2, — ), to the data of the key frame kfi The key frame start hour entry Tkf and the 
interpolation time length D are made to include. The elapsed time tm from the start of a musical 
piece performance is collated with this start time Tkf for every drawing, interpolation operation is 
performed one by one within this time length D, and if a musical piece performance reaches the 
following key frame, interpolation operation within the next time length D will be started. 
[0142] The timing diagram for explaining to drawing 21 the case where interpolation control by 
such time collating is performed is shown. Moreover, in drawing 22 , the example of a processing 
flow of the important section in this interpolation control is shown as "interpolation processing 
(2)", and processing at each step in this processing flow SI is as follows in it. 
[0143] [Step SI 1] If CG picture object moving part to which the interpolation flag which shows 
that this interpolation processing (2) is started by interruption of the predetermined time interval 
set up according to the throughput of a system, and it is under interpolation is stood is detected, 
first, in a step SI 1, data required for interpolation control of performance information, control 
information, etc. will be acquired about the moving part concerned, then it will progress to a step 
SI 2. 

[Step SI 2] At a step SI 2, the elapsed time tm from the performance start concerned is 
acquired from that performance information when being directed by the reproduction pointer, and 
it is this elapsed time tm The following key frame kfi+1 It collates with start time Tkf, when 
elapsed time tm has reached at the key frame start time Tkf (YES:tm>=Tkf), it progresses to a 
step SI 5, and when that is not right (NO:tm<Tkf), it progresses to a step SI 3. 
[0144] [Step SI 3] At a step SI 3, to the multiplication value (the amount of total displacement : 
an angle of rotation and a travel) of the velocity value V of performance information, and the 
scale value Vs of operation If key frame movement magnitude to the last position from the 
starting position of the key frame kfi concerned obtained by carrying out the multiplication of the 
suitable arbitrary coefficients Ki further is made into Ai-KixVxVs About the moving part 
concerned, by Aix[(Tkf-tm )/D} = "interpolation variation" Vm, the interpolation variation Vm to a 
interpolation position is calculated from a starting position this time, and it progresses to a step 
SI 4. In addition, as for a coefficient Ki, carrying out to less than one is desirable so that the 
total interpolation section from the current position to the purpose position may cover 1 key 
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frame period. 

[Step SI 4] After drawing in the interpolation position at a step SI 4 this time which displaced 
only the interpolation variation Vm from the starting position and moving moving part to this 
position from the last interpolation position, a return is carried out, if there is the following 
moving part to which the interpolation flag is stood, it will return to a step SI 1 and processing 
same about the following moving part will be performed, and if there is nothing, it will return to 
the state stand by the next starting. 

[0145] [Step SI 5] At a step SI 5, the key frame kfi is updated ("kfi+1" ->kfi), start time Tkf is 
updated ("Tkf+D" ->Tkf), and it is the following key frame kfi+1. After asking for a starting 
position, it progresses to Steps SI3 and SI4. 

[0146] According to this interpolation processing (2), according to the time tm corresponding to 
the periodical interruption permitted according to the throughput of a system, the interpolation 
position in a key frame at that time is certainly called for so that I may be understood from the 
above steps SI1-SI5. Moreover, especially this interpolation processing (2) is suitable to acquire 
CG animation picture which synchronizes with being answered by event performance information. 
As mentioned above, the picture generation method from which the animation which operation of 
a smooth picture was secured by interpolation processing of this invention, and moreover 
synchronized with the musical performance is obtained is realizable. 

[0147] [Positioning motion control by performance data analysis] to musical piece performance 
information like MIDI performance information Since the data for ** and various performances, 
such as events (note-on / OFF, various control information, etc.) illustrated in the area DA and 
CA of drawin g 8 , and time, tempo, a program change (tone selection), are contained By treating 
as a thing in connection with control of the whole picture it not only uses these performance 
data for each operation of picture object moving part, but For example, a more advanced and 
colorful dynamic image is generable by analyzing as what gives the special information about 
**** and the performance form of a performance model, or using as what gives specific picture 
control-lead information. 

[0148] Then, if this invention is followed, using the coordinate generation algorithm which 
analyzes a settlement of performance data, the coordinate data after movement of a picture 
object (CG model) will be generated, and the method of controlling operation of a picture object 
based on this coordinate data will be offered. In this method, as shown in the conceptual diagram 
of drawing 23 , the amount of many required for move control of the coordinate value of each 
part of CG model or an angle value is calculated from the performance data (for example, events, 
such as note-on / OFF) given from the musical piece information source MS by the coordinate 
generation algorithm PA which constitutes some of musical sound and picture generation 
modules, and the value acquired by this calculation is changed into CG data expressed with a 
key frame coordinate value etc. And synchronizing with generation of the musical sound based 
on performance data, natural performance operation of CG model is appeared by carrying out 
musical-sound generation based on CG data. 

[0149] If an example is given, a more natural picture is generable by analyzing two or more 
performance data and controlling the movement of the predetermined moving part of a picture 
object like previous statement, using the analysis result. For example, the moving part which 
plays the musical instrument can be positioned more with nature by dividing two or more events 
on a certain time-axis, and presuming the performance form at that time from a set of the note 
number of "note-on" information. It becomes possible to analyze a chord from a performance 
data distribution, speaking of the example which generates the pianist who is playing the piano as 
a picture object, to carry out as point to point control of the position of a pianist's wrist is 
carried out based on this analysis result, and to realize a pianist's natural movement. 
[0150] In order to realize such natural movement, a coordinate generation algorithm is used, the 
performance form of the player model which analyzes a settlement of performance data and 
plays a musical instrument is presumed, and after calculating the coordinate value of the position 
which should be moved according to this presumption as CG data, performance operation of a 
player model is controlled by this invention with this CG data. 

[0151] In addition, in order to appear such performance operation often [ precision ] and with 
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reality Although to analyze much performance data according to various operations or what is 
necessary is just made to perform the guess which also considered the context of performance 
data if needed, in this case It is desirable to perform analysis or a guess beforehand, to create 
the motion-control information on a player model for the positive synchronization with musical- 
sound generation, to use this motion-control information at the time of a musical piece 
performance, and to reproduce performance operation so that it may mention later. Positioning 
motion control by performance data analysis [0152] [wrist positioning processing] — **** which 
can realize motion control which analyzes performance data, presumes a performance form first 
according to this invention, and CG model positions also on real time — I will explain the easy 
motion-control method This method makes an example the case where the wrist is positioned by 
making the player model of keyboards, such as a piano like an above-mentioned pianist, into a 
picture object, and is named "wrist positioning processing" for convenience. In this "wrist 
positioning processing", the note-on data of the same timing are used as a settlement of 
performance data, and the position of the wrist of the player model which plays the keyboard 
from these data is computed. The rough plan for explaining theoretically the wrist positioning 
processing (SW) mentioned above is shown, and when the keyboard KB formed along with the X- 
axis of XY flat surface is seen from a top (positive side of the Z-axis), the physical relationship 
over the keyboard KB of the left-hand neck WR of the player model by which CG drawing 
processing is carried out is expressed to drawing 2 4 . 

[0153] Moreover, the flow chart which shows roughly the coordinate computational algorithm of 
wrist positioning processing (SW) is shown in drawin g 25 . The processing flow SW shown in this 
drawing can be started, when the performance data about the moving part related to a wrist are 
received and two or more note-on data nickel belonging to the timing it can consider that is 
almost the same among these performance data (these note-on data shall express with a note 
number nickel) come. It is as follows if processing at each step in this processing flow SW is 
explained. 

[0154] [Step SW1] In Step SW1, the note-on data nickel of these same timing are detected 
altogether, then progress to Step SW2. 

[Step SW2] At Step SW2, a value "nickel-No" is compared with a value "0" about all the same 
timing note-on data nickel detected at Step SW1. Here, No is the note number of the 
predetermined note chosen as a criteria position, and a majority logic is adopted as the judgment 
of this comparison to two or more values "nickel." And as a result of this comparison, if judged 
with nickel-No>=0 (YES), it progresses to Step SW6, and when that is not right (NO:nickel- 
No<0), it will progress to Step SW3. 

[0155] [Step SW3] When it progresses to Step SW3, the detected same timing note-on data 
nickel are recognized to be a thing accompanied by the performance form flipped on the left of a 
pianist, and progress to Step SW4. 

[Step SW4] At Step SW4, about these same timing note-on data nickel, the average NL of a 
value "nickel-No" «0) is computed, and it progresses to Step SW5 further. 

[Step SW5] As what expresses with Step SW5 the position of the left-hand neck WL on the line 
coordinates to which the average NL of these same timing note-on ** made the position of a 
note number No the zero, after performing CG drawing of the left-hand neck WL in this position, 
a return is carried out and it waits for arrival of the same following timing note-on data. 
[0156] [Step SW6] When it progresses to Step SW6, the same timing note-on data nickel are 
recognized to be what is flipped on the right of a pianist, and progress to Step SW7. 
[Step SW7] At Step SW7, about these same timing note-on data nickel, the average NR of a 
value "nickel-No" (>=0) is computed, and it progresses to Step SW8 further. 
[Step SW8] At Step SW8, after performing CG drawing as a position of the right-hand neck WR 
on the line coordinates which made the position of a note number No the zero for the average 
NR of these same timing note-on **, a return is carried out and it waits for arrival of the same 
following timing note-on data. 

[0157] When it progresses to Step SW6 after passing as a result of [ SW1-SW5 ] such 
processing (for example, steps), as shown in drawjng.24 , CG drawing of the left-hand neck WL is 
carried out in the position of the average NL «0) on the rectilinear coordinate system (X-axis) 
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which makes the position of a note number No a zero. 

[0158] Thus, if the positions NL and NR of right-and-left both the wrists WL and WR are 
determined, it will become possible to be also able to determine the position of an elbow (elbow), 
an arm (arm), and a shoulder automatically, and to determine the frame of **** of a player 
model. 

[0159] In the example of the wrist positioning processing mentioned above, the averages NL and 
NR of the same timing note-on data nickel should be computed, and the wrist position shall be 
determined, only using this average. Carrying out a deer, various kinds of presumption and 
operations can be performed, the movement of each part of a player model can be controlled 
based on this, and a player model can be further operated with nature. 

[0160] For example, as shown in the right-hand side of dra_yyjngJ24 , in the case of the right-hand 
neck WR, among the same timing note-on data nickel, a value "nickel-No" is equivalent to a 
little finger, and the thing with this smallest value is possible [ a value / the largest thing ] also 
for equivalent to the thumb, then presuming and making this presumption reflect in the 
movement of each part of a player model. Furthermore, since the length of both fingers differs in 
this case, you may calculate performing weighting according to the ratio of the length of both 
fingers etc. about a wrist position. 

[0161] Moreover, in an above-mentioned example, when the musical instrument played is organ, 
for example although it is made to play an one-step type keyboard instrument like a piano as 
shown by the keyboard KB which meets one rectilinear coordinate system (X) in drawing 24 , two 
steps of upper and lower sides can be prepared as a rectilinear coordinate system. The 
movement of each part of a player model is controllable like an one-step type keyboard 
instrument by preparing the organ performance algorithm for determining each wrist position to 
each stage system of coordinates according to performance data, such as assigning an upper 
case side to a right hand and assigning a lower-berth side to a left hand also in this case, 
respectively. 

[0162] [Point to point control by the performance data analysis which uses a read ahead 
together] As mentioned above, by creating motion-control information beforehand, the control 
which positions by analyzing performance data like this wrist positioning processing, and 
presuming a performance form can be certainly synchronized with the performance of musical 
sound, and can be realized with a sufficient precision. That is, by giving and analyzing various 
operations and guesses to the performance data constellation obtained by read ahead 
beforehand, the position where the moving part of a picture object like the player model which 
plays the musical instrument is natural is predicted, and operation of a picture object is 
controlled using the prediction result in analysis to the musical piece and picture generate time 
corresponding to the performance data constellation concerned. If it does in this way, in case 
the information on a wrist position etc. is created by wrist positioning processing, creation, 
presumption mentioned above, and the operation of the positional information of remaining each 
part (an elbow, an arm, shoulder, etc.) can also be beforehand created with a margin, for example, 
without being late for a musical-sound performance. Therefore, to a musical piece and a picture 
generate time, it is generable, synchronizing a much more advanced picture with a musical-sound 
performance certainly. 

[0163] The case where "wrist positioning processing" shown in drawing 25 using such read- 
ahead analysis is performed is explained. In this case, what is necessary is to make most 
processing flows SW of drawing„2_5 correspond to the read-ahead pointer processing step SE 12 
of drawin g 18 (A), and just to make only the drawing processing in Steps SW5 and SW8 
correspond to the reproduction pointer processing steps SE23 and SE24 of drawing 1 8 (B). 
[0164] That is, if the performance data directed by the read-ahead pointer PP at the step SE 1 
of read-ahead pointer processing of drawing 18 (A) are detected one by one, it will shift to Step 
SW1. At this step SW1. after detecting altogether the note-on data nickel of the timing it can 
consider that is almost the same and passing through Step SW2 from these performance data, it 
progresses to Step SW5 or Step SW8 one by one through Steps SW3 and SW4 or Steps SW6 
and SW7. 

[0165] In Steps SW5 and SW8, it stores in storage with the pointer as CG data showing the 
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position of the wrists WL and WR on the line coordinates of "nickel-No" computed about the 
group of the same timing note-on data nickel which made the position of a note number No the 
zero for Averages NL and NR f and the prior performance data analysis processing based on a 
read ahead is finished here. And the drawing processing in Steps SW5 and SW8 is made to 
correspond to the reproduction pointer processing steps SE23 and SE24 of drawing 18 (B) at a 
musical piece and a picture generate time. That is, after reading CG data of Wrists WL and WR 
which correspond CG data for wrist positioning corresponding to directions of the reproduction 
pointer RP from storage at a step SE 23, in a step SE 24, based on this CG data, the point of 
Averages NL and NR is made into the position of Wrists WL and WR as a note zero (No), and CG 
drawing is performed. 

[0166] [Display change of CG model] Further, to musical piece information, since the 
performance data in which various use like for example, a program change other than the 
performance data used in the old example is possible are contained, if picture control is 
performed using such information, a much more colorful dynamic image is generable. If the 
example of 1 use is given, performance data, such as a program change, can be used as picture 
control information for a display change of the CG model IM. In this case, what is necessary is to 
prepare two or more sets of the coordinate generation (positioning) algorithms [ PA / PA and / 
2 ] 1 peculiar to each musical instrument and — corresponding to the CG models [ IM / IM 
and / 2 ] 1 and — which play a specific musical instrument as the CG model IM and a positioning 
algorithm PA, and these models, respectively, and just to make it change to corresponding CG 
model and a corresponding positioning algorithm using the program change information that it is 
used for tone selection. 

[0167] That is, as shown in the conceptual diagram of draw ing 26 , the specific performance data 
of the musical piece information source MS judge the information about which musical 
instrument is offered, and choose two or more CG model algorithm group IM1-PA1 and IM2-PA2 
which were prepared beforehand, CG model of — which corresponds from inside, and an 
algorithm based on the judgment result of this musical instrument classification. While changing 
to the musical instrument picture and player model to which the CG model IM for drawing was. 
directed if tone change directions are obtained using the tone information in performance data 
when it follows, for example, the tone information on a program change is used as specific 
performance data, the coordinate generation algorithm PA to perform can be changed and can 
carry out CG drawing processing of the player model based on a corresponding algorithm. 
[0168] For example, if draw ing 24 , a piano performance algorithm as shown in the example of 25, 
and the organ performance algorithm mentioned above are explained These piano performance 
algorithms and the organ performance algorithm are programmed to answer the piano tone 
information and organ tone information in performance data, respectively. When the player model 
which plays an one-step type keyboard instrument slack piano based on a piano performance 
algorithm is drawn when the tone information on performance data is a piano, and tone 
information becomes organ, with these tone change directions What is necessary is just to draw 
the player model which changes an algorithm to an organ performance algorithm and performs 
this organ, while changing the picture which should draw into the organ which is a two-step type 
keyboard instrument. 

[0169] In addition, the CG model IM and Algorithm PA are chosen in a selection signal, and it can 
also be made to change and display on arbitrary musical instrument performance pictures by 
preparing a musical instrument or an algorithm selection button in a user interface UI, and 
operating this selection button arbitrarily, as the dashed line of drawing 26 shows. 
[0170] 

[Effect of the Invention] As mentioned above, the operational parameter for controlling the 
movement of each part of the picture object displayed on a display screen corresponding to the 
musical piece which should be performed according to this invention is set up beforehand. Since 
it is made to generate the picture by which the movement of each part was controlled according 
to the operational parameter set up based on corresponding musical-sound control information 
and a corresponding synchronizing signal at the time of the performance of the musical piece 
concerned The picture generated can change in one with this according to advance of not only 
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**** of the music performed but a performance. 

[0171] In this invention, since it has the parameter setting mode in which the operational 
parameter of each moving part of a picture object was set up arbitrarily, a participation [ whose 
user the dynamic image which was only excellent in a sense of togetherness with music can be 
danced based on performance data it not only carries out graphic display, but, and can set up 
the movement of a picture object like a hand freely ] type man machine interface can be offered. 

[0172] It can synchronize certainly to generation of musical sound by using CG data for which 
drawing at the time of event generating (at the time of reproduction) was prepared since 
performance data were analyzed in advance by read-ahead analysis processing according to this 
invention and CG data were prepared beforehand, can perform, is sufficient also for drawing 
delay or ", and is hard " coming to be generated. Moreover, it becomes possible to generate 
generous CG picture as the hand of not being directly related to the event concerned is made to 
be raised by Pianist CG in a suitable distance, for example, since the drawing processing burden 
at the time of reproduction is mitigated. 

[0173] According to interpolation processing of this invention, since interpolation control 
according to the throughput of a picture generative system is performed using the key frame 
corresponding to the synchronizing signal, operation of a smooth picture can be secured and, 
moreover, the animation which synchronized with the musical performance can be obtained 
certainly. 

[0174] Furthermore, according to this invention, the animation which operates a player model 
with reality with a natural performance form can be created by analyzing a settlement of 
musical-sound data and predicting a performance state. Moreover, colorful animation can be 
easily changed by preparing two or more algorithms of such analysis processing selectable 
corresponding to various pictures. 

[0175] And altogether, since the performance data and the synchronizing signal of musical piece 
information which are simultaneously performed as a musical sound are used for generating CG 
dynamic image in this invention, the movement of a picture is peculiar to each musical piece « 
performed, and fits it, and it differs for every musical piece in it, and, moreover, the animation 
which synchronized with the musical piece performance can be easily created to it. 
[0176] In this invention, the operational parameter set up according to the musical piece in 
parameter setting mode can be memorized to a storage like a floppy disk, and an operational 
parameter can be read from this storage according to the musical piece performed at the time of 
a musical piece performance. 



[Translation done.] 



http://www4.ipdl jpo.gojp/cgi-bin/tran_web,cgi_ejje 



03/03/06 



1/2 s<— v 



* NOTICES * 

Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] Drawing 1 is the block diagram showing the hardware composition of the musical- 
sound response picture generative system by one example of this invention. 
[ Drawing 2] Drawing 2 is the block diagram showing the software configuration of the musical- 
sound response picture generative system by one example of this invention. 
[Drawing 3] Drawing 3 shows the example of a picture displayed on a display screen in dance 
mode. 

[ Drawing 4] Drawing 4 is drawing showing the display structure of a picture object (dancer) in a 
**** outline. 

[Drawing 5] Drawing 5 is drawing showing the outline of the configuration procedure performed in 
the dancer setting mode by the musical-sound response picture generation method by one 
example of this invention. 

[Drawing 6] Drawing 6 is drawing showing the "dancer setting" dialog screen in dancer setting 
mode. 

[Drawing 7] Drawing 7 is drawing showing the "channel setting" dialog screen in dancer setting 
mode. 

[Drawing 8] Drawing 8 is drawing showing the "data selection" dialog screen in dancer setting 
mode. 

[Drawing 9] Drawing 9 is drawing showing the "operation setting of arm" dialog screen in dancer 
setting mode. 

[Drawing 10] Drawing 10 is drawing showing the "operation setting of leg" dialog screen in 
dancer setting mode. 

[Drawing 11] Drawing 11 is drawing showing the main functional slack dance module DM of the 
**** module I. 

[Drawing 12] Drawing 12 is drawing showing the performance data-processing flow in dance 
mode. 

[Drawing 13] It is rough drawing. **** for explaining operation of the picture object (dancer) in 
dance mode when drawing 13 (a) sets up right-and-left individual operation — drawing 13 (b) 
**** for explaining operation of the picture object (dancer) in the dance mode at the time of 
setting up axial-symmetry operation — **** of the picture object (dancer) for being rough 
drawing and drawing 13 (c) explaining the attenuation processing in dance mode — it is rough 
drawing 

[Drawing 14] Drawing 14 is drawing showing the beat processing flow in dance mode. 
[Drawing 15] Drawing 15 is drawing showing the attenuation processing flow in dance mode. 
[Drawin g 16 ] Drawing 16 is drawing showing another attenuation processing flow in dance mode. 
[Drawing 17] Drawing 17 is the conceptual diagram showing theoretically the read-ahead analysis 
processing by this invention. 

[Drawing 18] Drawing 18 is drawing showing the read-ahead analysis processing flow by one 
example of this invention, and (A) and (B) express processing with a read-ahead pointer and a 
reproduction pointer, respectively. 

[Drawing 19] Drawing 19 is drawing showing the timing diagram for explaining "number-of-times 



http://www4.ipdl.jpo.gojp/cgi-bin/tran_web_cgLejje 



03/03/06 



2/2 ^— v 



control of interpolation in the appointed time length" by this invention. 

[Drawin g 20} Drawing 20. is drawing showing the processing flow of "number-o1H:imes control of 
interpolation in the appointed time length" by this invention. 

[Drawing 21] Drawing 21 is drawing showing the timing diagram for explaining "interpolation 
control by time collating" by this invention. 

[Drawing 22 ] Drawing 22 is drawing showing the processing flow of "interpolation control by time 
collating" by this invention. 

[Drawing 23] Drawing 23 is a conceptual diagram for explaining "the point to point control by 
performance data analysis" by this invention. 

[Drawing 24] Draw ing 24 is drawing showing the timing diagram for explaining "wrist positioning 
processing" by this invention. 

[Drawing 25] Drawing 25 is drawing showing the processing flow of "wrist positioning processing' 
by this invention. 

[Drawing 26] Dr awin g 26 is a conceptual diagram for explaining the display change of CG model 
by this invention. 
[Description of Notations] 

1 CPU (Central Processing Unit), 

2 ROM (Read Only Memory), 

3 RAM (RAMX 

4 Input Unit, 

5 External Storage, 

6 Input Interface (I/F), 

7 Sound-Source Equipment, 

8 Display-Processing Equipment, 

9 Bus, 

S Sequencer module, 
A Sound-source module, 
I **** module, 

PS Parameter setting sub module, 

SP Sound system SP containing the musical-sound signal processor 10 and a loudspeaker 1 1 

DP Display system including display-processing equipment 8 and a display 12, 

D 3-dimensional picture object (dancer), 

DM Dance module, 

DA "Data-type" setting area, 

NS "Note-on" setting section, 

CS "Control" selection setting section, 

BS "Beat type" selection setting section, 

CA "Channel selection" setting area, 

BR "Beat output-value" setting display, 

RR "Of operation attenuation value" setting display, 

SR "Of operation scale" setting display, 

PP Read-ahead pointer, 

RP Reproduction pointer, 

kfi, kfi+1, — Key frame, 

MS Musical piece information source, 

PA;PA1, PA2, — Coordinate generation algorithm, 

KB Keyboard 

WL Left-hand neck, 

KR Right-hand neck, 

IM;IM1, IM2, — CG model, 

UI User interface. 



[Translation done.] 
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